Is Nvidia Jetson product family also suitable for machine learing model training?

Training a neural network involves a lot more computation than inferencing from a pre-trained one. During training, a neural network must go through both forward propagation and back propagation steps. The back propagation step is computationally expensive, since it requires many gradient computations and updates to weight variables.

During inference, the only computation needed is forward propagation, where the input is multiplied to the weights to produce a prediction. Boards like the NVIDIA Jetson are optimized for fast matrix multiplication, hence why they are effective during inference but not as much for training.

Another reason for the Jetson's popularity is its small form factor. Architects choose it because it can fit within the size and price requirements of the product while also offering decent performance. If you had unlimited budget and can fit a GTX Titan in your product for inferencing, you would do that, right? I worked in the autonomous vehicle sector for a while and that there is no way we could fit a machine with any full-size GPU in a car. There are no size limitations for training models, so naturally you would go after the machines with more powerful and efficient GPUs.