Meanwhile Linux will happily copy thousands of files per second, all on the same hardware.Įxplorer might be using incorrect ways to do I/O, but if that is the case, then that would mean that the proprietary, only-ever-developed-for-Windows-by-Microsoft tool that is also the main way users do I/O doesn't do I/O properly. Copying a bunch of small files on Windows takes forever, maybe doing around a few hundred files per second. The same operation on Linux is practically instant. You can see the list of detected or mounted Ext partitions from the Paragon ExtFS system tray icon context-menu. This is a very clever software and prevents mounting of the Linux swap partitions. Explorer will take somewhere from a few seconds to ~tens of seconds to list the contents of a SSD-backed directory with a couple thousand files. As you launch Paragon ExtFS, it will automatically detect existing Ext partitions and mount them in Windows. With Paragon ExtFS for Windows, you can work with a Linux native file. On top of that, when accessing many/all files in a folder in Linux, this gives you the inode, and if you sort your accesses by inode, throughput can rise to the ninth power on hard drives. Paragon ExtFS for Windows is a special utility which allows you to get full access to Ext2/Ext3/Ext4 file system in Windows. Os.scandir in Python was designed to encapsulate "listdir replacement APIs" and uses FindFirstFile/FindNextFile on Windows, and also allows most stuff to be done without an extra per-file stat() syscall on Linux. Worse, many modern cross-platform languages (including Python) don't expose `FindFirstFileEx` without platform-specific add-ons, which means that devs have no easy way to do Windows metadata access right even if they know there's one specific way to do it. ![]() Unfortunately, most code ported to Windows from Linux uses `GetFileAttributesEx`, under the assumption that it's as fast as `stat` is on Linux, but results in a poor user experience. Metadata performance is much better if the code uses `FindFirstFileEx`, which IIRC has no direct Linux equivalent, to grab metadata en masse for all files in a folder and then filtering the results down the files that the code needs to know about. Windows metadata performance is horrendously bad for code that needs to get metadata for a large number files in the same folder the way it would be done on Linux: by calling the Windows equivalent of `stat` (`GetFileAttributesEx`) for every file. It's a lot slower than it needs to be, especially for software ported from Linux (or POSIX-like environments in general), because metadata access has to be done in a Windows way to be performant. Windows filesystem metadata performance is also highly dependent on how applications access metadata.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |