want to hack on the actual hardware in my hands I want to bring my own Blackhole and show it off Want to bring Blackholes together and connect them at 800Gbps A 2D torus with 4 people, a 3D torus with 8 people? That's incredible! (1) Might be doable with effort. TT is freedum! ※ The image on the right was generated by Gemini Nano Banana (1)
NVMe SSD 2. M.2 NVMe PCIe 3.0 x4 Adapter : ADT-Link R42UF 3. 1000W ATX 3.1 Power Supply: Thermalright TR-TPFX-1000-W Purchased the entire set during Aliexpress's July sale for ¥32,335 DIY was cheaper than buying an eGPU box Reference sites https://darekasan-net.hatenablog.com/entry/2024/09/04/152918
Intel Core Ultra 7 155H 64MB Ubuntu 24.04.3 bare metal installed BIOS: Thunderbolt 3 -> Security Level: No Security Security -> Virtualization -> VT- d Feature: Disable ※ For some reason, in my case, vLLM wouldn't work and threw errors when IOMMU (VT-d) was enabled!? ※
worked smoothly. (request-venv) hayate@midnight:~/git/tt-inference-server$ curl -sS "http://localhost:8000/v1/completions" -H "Content-Type: application/json" -H "Authorization: Bearer $VLLM_API_KEY" -d "{ \"model\": \"meta-llama/$MODEL\", \"prompt\": \"Jim Keller is?\", \"max_tokens\": 60, \"temperature\": 0 }" | jq { "id": "cmpl-9c65c696ebaa4031a5900aaec091ab11", "object": "text_completion", "created": 1761145166, "model": "meta-llama/Llama-3.1-8B-Instruct", "choices": [ { "index": 0, "text": " (Part 2)\nJim Keller is a renowned American computer architect and engineer, best known for his work at AMD and Apple. He is credited with designing the x86-64 architecture, which is the foundation of modern personal computers.\nKeller's career spans over three decades, with significant contributions to", "logprobs": null, "finish_reason": "length", "stop_reason": null, "prompt_logprobs": null } ], "usage": { "prompt_tokens": 5, "total_tokens": 65, "completion_tokens": 60, "prompt_tokens_details": null } } https://docs.tenstorrent.com/getting-started/vLLM-servers.html#deploying-llms ※: For tt-inference-server branches, try bh-getting-started first, then proceed to dev if successful
+ p100a Now able to go out with my Blackhole anytime, anywhere Future work Investigate performance degradation from Thunderbolt connection (Is 8.0 Gb/s sufficient?) Evaluate Blackhole Peer to Peer 800Gbps connection performance Requires two or more P150 units. Yes, I want them! (2) pci 0000:52:00.0: 8.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x4 link at 0000:00:07.2 (capable of 504.112 Gb/s with 32.0 GT/s PCIe x16 link) 00:07.2 PCI bridge: Intel Corporation Meteor Lake-P Thunderbolt 4 PCI Express Root Port #2 (rev 02) (2)
to recognize Thunderbolt devices on connection /etc/udev/rules.d/99-removable.rules ACTION==“add”, SUBSYSTEM==“thunderbolt”, ATTR{authorized}==‘0’, ATTR{authorized}=“1” ※Reference URL: https://wiki.archlinux.org/title/Thunderbolt 2. Connect the p100a and verify with lspci Verify with lspci -vv -d 1e52:* 52:00.0 Processing accelerators: Tenstorrent Inc Blackhole The device must be displayed and three Memory regions (0, 2, 4) must be allocated 3. Re-apply hugepages (mandatory for plug-and-play connections) If the device shows up in lspci, manually run sudo /opt/tenstorrent/bin/hugepage-setup.sh If it displays Node 0 hugepages after: 4 , it's OK. You can also check the info with cat /proc/meminfo
tenstorrent_japan
malarkey
hatefulcrawdad
addyosmani
jacobian
bluesmoon
smashingmag
eileencodes
bermonpainter
matthewcrist
palkan
denniskardys
productmarketing
