Difference in opening a file in Python in Linux versus Windows

07
2014-07
  • davelupt

    What are the differences, if any, in how Python deals with opening/closing a file in Windows versus doing the same in Linux? I understand that in Linux the kernel acts as an interface between the software and the data stored in a hardware block device. Does Windows primarily do the same thing, or is there something under the hood that Python does to ensure the information is interpreted properly?

    • Answers
  • user55325

    On Windows, the CreateFile API call is used to open or create a file. This returns a file handle, which is analogous to a file descriptor on Linux. From that perspective, it works more or less the same way.

    On the other hand, as a Python programmer you have to be aware that Python does do some things differently that is not related to the low-level implementation. For example, on Linux opening the file in binary mode works the same way as text mode, but on Windows, where everything expects CRLF line endings, if you open the file in text mode, Python silently converts your \n to \r\n. This is obviously a bad thing if you are actually writing binary data. This is all quite well documented, though, so you don't need to worry about kernel-level details.


    • Related Question

    filesystems - What is a good file system for removable storage on Linux?
  • Michael Ekstrand

    I have a 500GB external drive I want to use on a couple of Linux systems, and looking for a filesystem for it. External drives are frequently formatted in FAT32, but I don't need to interoperate with Windows and would rather avoid the ugly limited kludge that is FAT.

    Since I only need to use it on Linux, I would use ext4 or XFS, but they store ownership information. Ideally, I'd use a proper Unix file system that doesn't track ownership (files are owned by whoever mounts the device, like they are when mounting a FAT32 partition), but I do not know of any file system that does that.

    What would be a good file system for this disk?


    • Related Answers
  • Adi

    You didn't mention anything about speed, but if you are concerned about it, then I recommend ext4. It is starting to mature nowadays.

    I would only use XFS on a large drive with large files.

    If you want stability and compatibility, then ext3 is the way to go.

    These articles and their comments may be of some use:

    http://www.phoronix.com/scan.php?page=article&item=linux_2632_fs&num=1

    http://www.debian-administration.org/articles/388

  • ubiquibacon

    FAT / FAT32 is typically only used for flash devices. These days FAT / FAT32 is a poor choice for any device compared to other available choices, and I often find the 4GB file size limit of FAT32 (less for FAT) very constraining. Don't use FAT / FAT32 unless you must (i.e. on a jump drive).

    Linux plays nicely with NTFS and EXT file systems, but Windows doesn't play nearly as nice with EXT. If you are 100% sure you will not need Windows interoperability then I would say go with EXT4, but if you think you may some day need to read the contents of your external hard drive from a Windows machine I would go with NTFS, other wise you will need a EXT viewer like EXT2READ.