Agner`s CPU blog

There is another way to reduce the TLB cost, which is used by the Mill architecture [1].

The TLB can be moved from the critical path of L1 cache read to DRAM read. Since DRAM reads are already slow, the TLB doesn't have to be fast, which simplifies it's design. However it means that all data on-chip are virtually addressed. Similarly to your proposal, all processes live in a single virtual address space, but the virtual/physical translation is retained.

The security problem is solved by using a PLB (Protection Lookaside Buffer) which is placed where TLB currently is. Since protection data is only needed to occasionally trigger an exception, it's not on a critical path of L1 read. Mill also employs so called well known regions, which are similar to per-thread/per-process segments and reduce the need to use the PLB in most cases.

Since many operating systems implement a memory mapping commands like linux's mmap, removing the virtual to physical translation would make it very difficult to port such OSs and its applications.

[1] millcomputing.com/docs/memory/ (circa 60th minute)