Custom board for running ML models

[kowshik1729]

Hi all,

I am planning to make a custom board for running ML algorithms. I decided to make this board besides having boards like Jetson Nano and Rpi4 is because they have many other peripherals that I don't need for my application. So, can someone please suggest me how to start off to make boards like these?? specifications I'm looking at are

1. A processor that can be burnt with Linux (Like ubuntu)

2. RAM - 4GB/8GB (depending upon the cost)

3. ROM - 16GB (can be eMMC or also SD card slot is enough)

[kksjunior]

Hi Kowshik,

While designing high speed digital circuits from scratch, you have to take care of numerous design rules and strategies which I don't want to elaborate.

For these kind of applications, usually to avoid design complexity and getting better time to market, we use SoMs (System on Modules). SoMs comes with different configurations with different Processor-RAM-ROM-GPU combinations. It's just a card kind of board with a Processor, RAM, Flash and an optional GPU and commonly they come with a 200 pin SODIMM male connector.

I always prefer Toradex for SoMs in my projects. You can have a look at their product lineup here - Browse Products - Toradex.

Their boards supports Windows CE, Linux(Yocto), Torizon etc. They provide you with all the BSPs, Drivers, Configuration tools etc.

They also provide carrier boards for prototyping in case you want to test your application while you design your custom board.

[salmanfaris]

@kowshik1729 I agree with what @kksjunior mentioned above, you can utilize the SoM, SoP's for your design, listing some of them.

1. Artemis Module - Low Power Machine Learning BLE Cortex-M4F
   If you are looking for SoC that can run ML, check the Artemis or nRF52840 .

more details: https://www.sparkfun.com/artemis

2. OSD335x-SM System-in-Package, Smallest AM335x Module
   The OSD335x-SM is the smallest Texas Instruments AM335x module. It integrates the TI Sitara ARM Cortex-A8 AM335x processor, DDR3 memory, TPS65217C PMIC, TL5209 LDO, all the needed passives, and 4KB of EEPROM into a single BGA package. At 21mm X 21mm, it is 60% smaller than the discrete components and 40% smaller than the OSD335x.
   link : http://octavosystems.com/octavo_products/osd335x-sm/ , https://beagleboard.org/pocket

3. Microchip SAMA5D2 ARM Cortex-A5 Processor 500 MHz
   Link: http://ww1.microchip.com/downloads/en/devicedoc/ds60001476b.pdf , https://www.microchip.com/design-centers/32-bit-mpus/microprocessors/sama5

[kowshik1729]

@kksjunior Thanks a lot for the reply. Yes, obviously there are a lot of rules, recommendations while making such kinds of boards. As you suggested, Is Toradex SoM optimized for running ML models specifically? because I've been looking for modules that can easily run the ML algos. And also, do I need to design a break out board for extending the pins of the SODIMM module?

[kksjunior]

@kowshik1729 Yes, a lot of different SoMs from Toradex are optimized for AI and ML based applications. They've developed their own Linux based "Torizon" OS for easy configuration and deployment. There are different SoMs available right from the one's having a normal NXP Vybrid SoC to the powerful ones like NXP i.Mx8 QuadMax and Nvidia Tegra K1. You also have options for embedded WiFi and BLE modules.

These are pure industrial grade modules with IT grade certification(-40° to +85° C), so although they're easy to work upon, I won't recommend them for hobby usage. There are lot of alternatives like @salmanfaris mentioned above if you're aiming for learning/hobby usage.

But if you're looking from a product perspective you can choose Toradex without hesitation.

See the below Links for AI & ML related applications with Toradex SoMs.
-> Webinar : Jumpstart cloud-connected computer vision and ML designs with Toradex.
-> Apalis AI Vision Starter Kit for the i.MX 8 Applications Processor
-> Rapid Creation and Deployment of ML Models at the Edge with Toradex AI Vision Starter Kit

Regarding the question on breakout board.

Yes you definitely have to design a "carrier board" which should hold the SODIMM module. There are a lot of readymade carrier boards for each family of SoMs directly available from Toradex which can be used for prototyping as well as for production. However as you've mentioned in your post about unwanted peripherals, it's always advised to design custom carrier boards for customized applications. Why do you need to pay for a couple of USB receptacles and ethernet connectors if you're not going to use them? Also a custom carrier board can always be designed according to the form factor and dimensions of your product design.

[kksjunior]

Posting something which I've forgot to mention in the first post.

The Raspberry Pi Compute Module is also an alternative to Toradex SoMs, though you don't have any configurations to choose. There is only one standard model available in every generation of the module.

Just like the Toradex SoM, even this guy comes with a DDR2 SODIMM connector.

More Details here.

[kowshik1729]

@kksjunior Thanks a lot for a very detailed explanation. I shall surely look into the options you've shown and dig more on this.

[kksjunior]

@kowshik1729 Keep digging untill you find the gold. All the best!

[kowshik1729]

@kksjunior Just a small naughty thought ....Is it possible to upgrade the RAM on those RPi modules? I mean if the modules has pin to pin compatibility then can I consider upgrading them? Or will I be having some limitations from the Broadcom SoC they have on board?

[kksjunior]

@kowshik1729 Haha, Nothing is impossible... Let's say if you get a pin compatible DRAM IC complying to BroadCom SoCs requirement without any limitations to the firmware running upon it you may be able to upgrade it. Provided that you didn't destroy the board while tinkering with it. The effort to result ratio will never match.
That's the reason why we get a wide variety of options from manufacturers like Toradex. All the Toradex SoMs belonging to the same device family are pin compatible. You can always upgrade or downgrade your computational module without disturbing the rest of your design!