AI Fabric Architect
Fal Ai
Hirezar Summary for South African Applicants
This fully remote full time position at Fal Ai may be open to South African applicants — check the source listing to confirm eligibility before applying. As a remote position, you can work from anywhere in South Africa — whether you're based in Johannesburg, Cape Town, Durban, or a smaller town.
Job Description
fal is the generative media ecosystem powering the next generation of AI products. We build the infrastructure, tools, and model access that teams need to move from idea to production, and do it at scale without compromise. For developers and enterprises, fal is the foundation that makes generative media not just possible, but practical: a unified platform where high-performance inference, orchestration, and observability come together to unlock new categories of AI-native products.
As generative media reshapes industries across a market projected to grow by hundreds of billions over the next decade, fal is becoming the ecosystem that ambitious teams build on.
About this role
You are the architect of the network that our GPU clusters run on. You own the fabric end to end, from the NIC in each GPU server through the spine and leaf across the AI fabric, and you know that in AI infrastructure the network is core to the machine. As fal brings its own GPU capacity online across multiple sites, you will define the fabric architecture every site is built from, and then personally make sure it gets built and performs. This is not an architecture-only role. We are a small team moving fast, and the right person is as comfortable writing a port map, tracing a bad optic, or tuning congestion control on the data center floor as they are setting technical direction.
What you will do:
* Own the fabric architecture for every site: GPU backend fabric (InfiniBand and/or RoCEv2), and GPU frontend Fabric
Own the fabric architecture for every site: GPU backend fabric (InfiniBand and/or RoCEv2), and GPU frontend Fabric
* Choose topologies and make the key calls: rail-optimized vs. other designs, InfiniBand vs. Ethernet, oversubscription ratios, and how the fabric scales from one pod to many
Choose topologies and make the key calls: rail-optimized vs. other designs, InfiniBand vs. Ethernet, oversubscription ratios, and how the fabric scales from one pod to many
* Turn the architecture into a repeatable network blueprint: cabling plans, port maps, fabric addressing and routing design, and a fabric bill of materials (switches, NICs, optics, cables)
Turn the architecture into a repeatable network blueprint: cabling plans, port maps, fabric addressing and routing design, and a fabric bill of materials (switches, NICs, optics, cables)
* Get hands-on with bring-up at each site: be on the floor for the first racks, validate cabling and firmware, configure the fabric manager and switches, and run performance acceptance (NCCL tests, burn-in) before capacity goes live
Get hands-on with bring-up at each site: be on the floor for the first racks, validate cabling and firmware, configure the fabric manager and switches, and run performance acceptance (NCCL tests, burn-in) before capacity goes live
* Tune for real workload performance: congestion control, PFC/ECN, adaptive routing, and NCCL settings for both inference and training traffic
Tune for real workload performance: congestion control, PFC/ECN, adaptive routing, and NCCL settings for both inference and training traffic
* Debug fabric problems wherever they show up, from a flapping link to a job that’s mysteriously slow
Debug fabric problems wherever they show up, from a flapping link to a job that’s mysteriously slow
* Evaluate and select AI Fabric vendors and hardware, and push back with data when something doesn’t meet spec
Evaluate and select AI Fabric vendors and hardware, and push back with data when something doesn’t meet spec
* Collaborate across automation and software to help build network monitoring, telemetry, and automation that let a small team run a large fleet, and be the top escalation point for AI Fabric incidents
Collaborate across automation and software to help build network monitoring, telemetry, and automation that let a small team run a large fleet, and be the top escalation point for AI Fabric incidents
* Document as you go so each new site is faster than the last, and help hire and mentor the network engineers who join after you
Document as you go so each new site is faster than the last, and help hire and mentor the network engineers who join after you
Qualifications:
* 7+ years in data center or HPC networking, including direct responsibility for production GPU cluster fabrics
7+ years in data center or HPC networking, including direct responsibility for production GPU cluster fabrics
* A track record of taking GPU fabrics from empty racks to production, with your own hands on the work, not only through vendors or integrators
A track record of taking GPU fabrics from empty racks to production, with your own hands on the work, not only through vendors or integrators
* Deep expertise in InfiniBand (NDR/XDR, UFM, subnet management) and/or RoCEv2 lossless Ethernet
Deep expertise in InfiniBand (NDR/XDR, UFM, subnet management) and/or RoCEv2 lossless Ethernet
* Strong command of rail-optimized and Clos/fat-tree designs, BGP/ECMP, and EVPN/VXLAN
Strong command
Tips for South African Applicants
Timezone Advantage
South Africa (SAST, UTC+2) overlaps well with European business hours and has a few hours of overlap with US East Coast. Mention your timezone flexibility in your application.
Salary in Context
Even without a listed salary, international remote roles typically pay 2-3x more than equivalent local positions in South Africa due to the exchange rate advantage.
Application Tips
Tailor your CV to international standards — use a clean format, highlight remote work experience, and include your English proficiency. Many SA applicants succeed by emphasising their strong work ethic and cultural adaptability.
Load Shedding Preparedness
If you're applying for a remote role, having a backup power solution (UPS, inverter, or generator) and mobile data as a backup internet connection shows employers you're prepared for South Africa's infrastructure challenges.
Related remote job searches
About Fal Ai
Fal Ai is a company in the Finance industry that hires remote workers from South Africa. They currently have 3 open positions on Hirezar. View all Fal Ai jobs →