BLU Discuss list archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Discuss] Whence distributed operating systems?
- Subject: [Discuss] Whence distributed operating systems?
- From: bogstad at pobox.com (Bill Bogstad)
- Date: Fri, 29 Apr 2016 14:33:54 -0400
- In-reply-to: <7f2a0608-5a3d-adb0-0f79-9644b7df7863@gmail.com>
- References: <chx4mavtvj8.fsf@iceland.freeshell.org> <20160421121138.GU12314@randomstring.org> <CAFq0N1z2xg2u0e6zbP8Aenyv1YAcW1C51vriQrj2V6XTh8KzpQ@mail.gmail.com> <571CD9C0.8030808@gmail.com> <CAJFsZ=r6YO9fW1js-h_C2JcUPi7fLCZiYzAHtq6P0L1D3Aw8GQ@mail.gmail.com> <3ba90ee1-0558-d121-a95d-a744fdbe32ff@gmail.com> <CAJFsZ=oc0e-Yj6zLjfxcrNVEHpBW2UuBKhAerw-qjjBy0ckhaA@mail.gmail.com> <7f2a0608-5a3d-adb0-0f79-9644b7df7863@gmail.com>
On Thu, Apr 28, 2016 at 10:41 PM, Rich Pieri <richard.pieri at gmail.com> wrote: > On 4/28/2016 8:30 PM, Bill Bogstad wrote: >> The fact >> that this was done via a fibre data bus vs. a faster local bus would >> seem to me to be an implementation detail. It still sounds like a >> NUMA with three levels of memory access (CPU local, QBB local, remote >> QBB). But all of the memory transparently visible in a program's >> address space. > > You might think that but it wasn't. The fibre bus never actually worked > well. It was too slow mixed with PCI, with too much latency. It got > congested and bogged down under even moderate loads. I still see that as an attempt at NUMA that didn't work because of the latency problems. Not really surprising actually. We all know what happens when a program/system tries to actually use more memory then is physically installed (i.e. actually using paging). The latency difference between local and remote QBB RAM might not have been as bad as between RAM and disk; but it sounds like it was more then enough to make it not work. I suspect that if they hadn't insisted on making the differences in latency completely invisible to applications, it is possible that it would have worked better. A way for a program to provide hints might have helped. Something like pin(address, length)/unpin(address, length). I think this would have been better from a programming perspective then having to explicitly manage moving data between local RAM and remote RAM/disk. You would still have a single flat memory space programming model which would always work (albeit slowly) and then you could throw in judicious pin()/unpin() function calls to help the OS keep active memory local. Bill Bogstad
- Follow-Ups:
- [Discuss] Whence distributed operating systems?
- From: richard.pieri at gmail.com (Rich Pieri)
- [Discuss] Whence distributed operating systems?
- References:
- [Discuss] Whence distributed operating systems?
- From: smallm at sdf.org (Mike Small)
- [Discuss] Whence distributed operating systems?
- From: dsr at randomstring.org (Dan Ritter)
- [Discuss] Whence distributed operating systems?
- From: jack at coats.org (Jack Coats)
- [Discuss] Whence distributed operating systems?
- From: richard.pieri at gmail.com (Rich Pieri)
- [Discuss] Whence distributed operating systems?
- From: bogstad at pobox.com (Bill Bogstad)
- [Discuss] Whence distributed operating systems?
- From: richard.pieri at gmail.com (Rich Pieri)
- [Discuss] Whence distributed operating systems?
- From: bogstad at pobox.com (Bill Bogstad)
- [Discuss] Whence distributed operating systems?
- From: richard.pieri at gmail.com (Rich Pieri)
- [Discuss] Whence distributed operating systems?
- Prev by Date: [Discuss] Looking for a replacement RSS/Atom feed reader extension for use with Firefox
- Next by Date: [Discuss] Whence distributed operating systems?
- Previous by thread: [Discuss] Whence distributed operating systems?
- Next by thread: [Discuss] Whence distributed operating systems?
- Index(es):