At the 2018 TensorFlow Dev Summit, Chris Lattner unveiled Swift for TensorFlow. The Google team published its first update to the public audience on their GitHub repository on April 27, 2018. Swift for TensorFlow, meanwhile, is still in its infancy. Additionally, it appears that it is too early for academics and developers to employ it in their work. Install the Swift for TensorFlow snapshot from the Swift official website if you still want to give it a try.

The Google-led project Swift for TensorFlow, which combined the TensorFlow machine learning library and Apple's Swift language, is no longer being worked on. It was archived by Google in February 2021. Nevertheless, several aspects of the project are still in use, such as Swift's language-differentiated programming. In this article, I will explain to you what happened to the project and what is still available for you to use. Just notice that for this article, I used a friend Mac for the test and he didn't appreciate it, I will tell you why after.

Swift for TensorFlow project shuts down

Why Swift for TensorFlow?

Swift for TensorFlow is a new way of developing machine learning models. It gives you the power of TensorFlow directly integrated into the Swift programming language.

The importance of machine learning paradigms justifies the need for top-notch language and compiler support. Calculating function derivatives to optimize parameters is a fundamental machine learning primitive known as gradient-based optimization. Using differential operators like gradient(of:) or differentiating against an integer model by invoking the gradient(in:) method are simple ways to differentiate functions with Swift for TensorFlow. These differentiation APIs are available for all kinds that follow the Differentiable protocol, including Float, Double, SIMD vectors, and your data structures. They are not restricted to Tensor-related notions. Here is a piece of code, it looks pretty nice:

// Custom differentiable type.
struct Model: Differentiable {
    var w: Float
    var b: Float
    func applied(to input: Float) -> Float {
        return w * input + b
    }
}

// Differentiate using `gradient(at:_:in:)`.
let model = Model(w: 4, b: 3)
let input: Float = 2
let (𝛁model, 𝛁input) = gradient(at: model, input) { model, input in
    model.applied(to: input)
}

print(𝛁model) // Model.TangentVector(w: 2.0, b: 1.0)
print(𝛁input) // 4.0

Here is a piece of code, Look how it is pretty nice:

The Swift for TensorFlow project includes a sophisticated toolchain in addition to derivatives to increase user productivity. Swift can be used interactively in a Jupyter Notebook to provide helpful autocomplete recommendations as you navigate a current deep-learning library's expansive API surface. In only a few seconds, you can start using your browser!

TensorFlow migration to Swift is incredibly simple because of Swift's strong Python interoperability, so you can simply call your preferred Python library with recognizable syntax, and you can convert your Python code gradually (or keep using your favourite Python libraries):

import TensorFlow
import Python

let np = Python.import("numpy")

let array = np.arange(100).reshape(10, 10)  // Create a 10x10 numpy array.
let tensor = Tensor<Float>(numpy: array)  // Seamless integration!

It looks like the story where perfect so what happened?

Swift for TensorFlow what happened?

Spoiler alert : I don't have the real answer cause, I am not yet in the TensorFlow Google Team but don't worry, it coming next. So this piece of my article will be just my view according to all my research on this subject. Let's start with some information about the S4TF project the war name of this project.

The project's GitHub repository states that it is currently in archive mode and won't be receiving any more changes. The project was promoted as a fresh approach to creating machine learning models, according to the repository. The next-generation machine learning platform Swift for TensorFlow was an experiment that included the most recent findings in machine learning, compilers, differentiable programming, systems architecture, and other areas.

By early evening on February 16 2021, neither Google nor the project's developers had responded to InfoWorld's queries on the project's status. However, supporters of Swift for TensorFlow listed some successes in the repo:

La nguage-integrated differentiated programming is being added to Swift, and work on this is still ongoing in the Swift compiler.
Development of a deep learning API focused on semantics and changeable values.
Enabling cutting-edge research using the SwiftFusion project that combines deep learning with probabilistic graphical models for 3D motion tracking and beyond.
Spinning off of numerous open source side projects that are currently under active development, such as Swift-Jupyter, PythonKit, and Swift-Benchmark. Swift-Jupyter allows users to use Swift within Jupyter notebooks.
There are other ideas for Swift's evolution, such as
- SE-0195: User-defined "dynamic member lookup" types,
- SE-0216: User-defined dynamically "callable" types,
- SE-0233: Make numeric refinement a new AdditiveArithmetic protocol, and
- SE-0253: Callable values of User-Defined Nominal Types.

Here comes my theory : As soon as Chris Lattner, the creator of Swift and a significant contributor to XLA, departed Google, this was going to happen. I still consider this to be a devastating loss, though. By including a neural engine in every device and making autodiffs a first-class citizen in their major development language, I believe Apple is missing a great opportunity. To be clear, I still believe that Swift + ML has a bright future, but I'm not sure if I trust Apple to care about the user experience of a good machine-learning API that serves the needs of those who don't want to wait 20 minutes for XCode to load their project. The question of whether Apple cares or not is further complicated by the fact that the company was willing to fork TF in order to assist Apple Silicon. There are undoubtedly some PMs who share my vision and others who have other goals, as with any huge organization.

Let now talk about the vestige of this love story :

S4TF Ressurection: running again on Google Colab :

I've been experimenting with sideloading Swift on Google Colab this last weekend, I used my friend Mac as said in the intro. The result in Swift for TensorFlow being loaded as a Swift package that has been pre-compiled as a binary target rather than a toolchain. I've reached the stage where I can compile and run any length of Swift code that I supply as a Python string.

The next step is to add SwiftPM support and install PythonKit in the Colab virtual machine. I stumbled found this tutorial when researching the Swift compiler and Swift Package Manager.

I decided to give Linux another shot after getting it to function on my bro Mac (he didn't appreciate that). I understood that using Google Colab is simpler to set up than Docker and that using Swift instead of Shell has some advantages (I could write a string literal to a file without learning the Shell command for that). Additionally, it was fascinating to watch Swift code in Google Colab perform a generally helpful task!

import swift 
swift.run(
''' import Foundation let fm = FileManager.default ... 
// do some stuff 
func doCommand(args: [String]) throws {
    let command = Process()
    command.executableURL = .init(fileURLWithPath: "/usr/bin/env")
    command.arguments = args
    try command.run()
    command.waitUntilExit()
}

try doCommand(args: ["swiftc", "-D", "DEBUG", "point.swift", "main.swift", "-o", "point-app"])
try doCommand(args: ["./point-app"])
''')

The link to the Colab notebook is provided here. The application can be copied and used by anyone. To pull from the save-1 branch of the GitHub repository, which will remain stable unlike the main branch, the first code block is updated.

Output :

debug mode Hello world! 4 20

A simple hello world but a real satisfaction. I make no promises that side-loading will eschew sending Python text in favour of supporting the entire Jupyter Notebook experience. However, the new Swift for TensorFlow can use cloud GPUs and TPUs thanks to sideloading. So just try and give me feedback in the comments. I will just give you a small overview of the great features I try to test.

Major Features

Automatic Reverse Differentiation (Forward not implemented yet)
Define-by-Run design (no sessions required)
Swift optimized to include machine learning specific functions
Allows Python APIs access in Pythonic way
Includes PyValue type for Python’s dynamic system behavior

Python Interoperability

With Swift for TensorFlow, we can use Python APIs in the most Pythonic way. To access Python APIs one has to import Python into the program as in the following example code snippet.

import Pythonlet np = Python.import("numpy")  // akin to `import numpy as np`
let pickle = Python.import("pickle")
let gzip = Python.import("gzip")

And Swift for TensorFlow also has a new type called PyValue which exhibits the complete dynamic type system behavior of Python in Swift without affecting the behavior of other types in Swift.

var x: PyValue = 3.14159
print(x * 2)  // Prints "6.28318"
x = "string"
print("now a " + x)  // Prints "now a string"

Automatic Differentiation

Swift for TensorFlow has built-in support for computing gradients of functions with respect to other variables. This functionality has been directly incorporated into Swift’s compiler for optimized behaviour. It supports two differential functions: #gradient(of:withRespectTo:) and #valueAndGradient(of:). Although it didn’t work for me 😩 (it’s too early) the documentation says the following syntax to follow.

@differentiable(reverse, adjoint: dTanh)
func tanh(_ x: Float) -> Float {
  // ... some super low-level assembly tanh implementation ...
}
func dTanh(x: Float, y: Float, seed: Float) -> Float {
  return (1.0 - (y * y)) * seed
}// Get the gradient function of tanh.
let dtanh_dx = #gradient(of: tanh)
dtanh_dx(2)
// Get the gradient function of foo with respect to the first parameter.
let dfoo_dx = #gradient(of: foo, withRespectTo: .0)
dfoo_dx(3, 4)

Working with Tensors

As a simple example, I will create a Tensor instance on which we apply some operations using TensorFlow.

import TensorFlowvar x = Tensor([[1, 2], [3, 4]])
for _ in 1...5 {
  x += x
}
print(x)  // Prints "[[32.0, 64.0], [96.0, 128.0]]"

Discussion

It appears that TensorFlow for Swift should have been used instead of Swift for TensorFlow. This is false because TensorFlow support has been added to the Swift compiler, making Swift more like a machine-learning language than just a wrapper for TensorFlow and Python libraries. Since Python is widely used in the machine learning and data science communities, it also offers access to Python APIs in a Pythonic way and features a new type for instances that behave like dynamic systems in Python, helping to maintain consistency in workflow.

TensorFlow, which was created by Google and made available as open source in 2015 and achieved in 2021, has possibly been less popular recently. Disney and Blue River Technology, two TensorFlow customers, switched to PyTorch instead of TensorFlow due to Facebook's PyTorch. PyTorch has been praised for its simplicity of usage.

Conclusion

This article taught us about using Swift for TensorFlow and how simple it is to do so because Swift is very similar to Python and appears to be a scripting language but is rather quick. We showed that using Python APIs is possible with Swift for TensorFlow and that Swift's compiler has been highly optimized to include support for automated differentiation, which is crucial for machine learning jobs. Additionally, we learned how to leverage TensorFlow with Swift and constructed our Tensor instances to play around with (using the fundamental operator +). Finally, the story behind this love story. So please tell me what you think about this article and the story tech telling. What is your opinion about this love story end?