Introduction

As AI technology continues to advance at a rapid pace, building a powerful server capable of handling demanding AI
workloads is becoming increasingly important. In this post, I’ll walk you through my journey of creating an
artificial intelligence (LLM) and Whisper dedicated server using an old Windows 2016 license and Ubuntu. This
setup will provide a robust platform for running AI models, processing large datasets, and integrating with other
tools.

Gathering the Components

To start, I had an old Windows Server 2016 license lying around, which provided the foundation for our server.
Additionally, I obtained a copy of Ubuntu 24.04 Server. I downloaded the LLM (Large Language Model) and Whisper AI models from their respective repositories.

Setting Up the Server

First, I installed Ubuntu on my old Windows Server hardware using a USB installer drive. The installation process
was straightforward, and I chose to install the 64-bit version of Ubuntu. Once installed, I configured the network
settings and set up the root password for the server.

Next, I enabled Secure Sockets Host (SSH) in Ubuntu to access the server remotely. This allowed me to control
the machine from anywhere, making it easier to manage and monitor the server.

Configuring the Server

To optimize the server for AI workloads, I made several configuration changes:

• CPU Affinity: To ensure efficient processing of AI models, I set CPU affinity for each model using the
  taskset command. This allowed me to allocate specific CPUs to each process. I have 24 total threads in my 1U Dual Xeon with 12 threads reserved for the Virtual Machine.
• Memory Allocation: I adjusted memory allocation settings to accommodate the increased demand from running
  multiple AI models simultaneously. 32 GB of DDR3 ECC RAM allocated to the VM.
• Network Settings: I optimized network settings to minimize latency and ensure efficient data transfer, and port forwarded nesseasry ports and application profiles in UFW.

Testing the Server

To test the server, I ran a few AI model simulations using LLM and Whisper. The results were impressive, with the
server handling demanding tasks with ease. The Ubuntu-based setup provided excellent performance and stability,
allowing me to focus on developing and refining my AI models.

Conclusion

In this post, we’ve explored how to create a powerful AI-ready server by combining an old Windows Server 2016
license with Ubuntu. By installing LLM and Whisper AI models, we were able to build a robust platform for
processing large datasets and integrating with other tools. This setup provides a solid foundation for anyone
looking to develop and deploy AI applications.

Future Development

In the future, I plan to explore further optimizations, such as:

• GPU Support: Adding a graphics processing unit (GPU) would significantly enhance performance for AI
  workloads.

• As AI continues to evolve, I’m excited to see where this setup takes me. Join me on this journey by following my
• future updates and exploring new possibilities in AI development!

---

Filed under: Uncategorized - @ April 26, 2024 7:32 pm