External file access
I apologize if this question has been answered somewhere and I just haven't found it in my searching.
In short: Can python scripts run through Pythonista access (non-script) files in places like Dropbox or iCloud?
To give a usage example, I am a researcher who uses python to perform data analysis on images. I mainly use my work computer for this, but I'm picturing a situation where I might want to do some last minute analysis to fix up a plot on the flight to a conference when I'd be off wifi, for example.
Say I have some image(s) in my Dropbox/iCloud (Also assume they've been downloaded to the device). Can I have my python script access files in either location for use in my script? If so, do I access these files a normal way (open(...) or similar), or using some other method - and will this method work offline? Let's say I generate a .dat file in the process - can I write that back to Dropbox/iCloud so I can get to it next time I'm on a computer?
I currently travel with a laptop, but it's certainly much easier to fly with my iPad. Pythonista looks like the perfect app to bridge the gap - this is the last thing I'm curious to know if it can do.
Phuket2 last edited by
Chilibird last edited by Chilibird
Hi @Phuket2 , thanks for your response! I ended up taking the plunge and downloading Pythonista. It seems like a pretty great app, but I'm definitely struggling with getting my script to open a file in Dropbox. I'm hoping maybe you or someone else reading this could give me a nudge in the right direction.
I can successfully import/open my processing script, called process.py, and I can run it. It makes it up to my first "f = np.loadtxt("./mydata.dat")" line and stops, saying it can't find that file. Fair enough - even though the script and files are in the same folder, I haven't imported the data files in to Pythonista yet.
So, I go back to the "external files" section of the browser and hit "open". I go to my dropbox and navigate to my folder (same folder as my script is in). All of my ".dat" files are greyed out and unselectable, and the blue cloud icon is next to them, also dimmed out and non-functional. Alright, those cloud icons make me think maybe I need to go in to the Dropbox app and tell it to make the files available offline, so I do, but nothing changes (I've tried closing/reopening both Pythonista and Dropbox, and restarting the iPad in case it was an issue of data not updating across apps). I've tried importing a whole folder with no success either. I just get an error saying the folder couldn't be imported.
I do some googling which suggests I need to use the "import" option under new files to make non-python files available, but I run in to the same not-selectable, greyed-out-with-cloud-icon.
Edit: I realized I can also use the files app to download the files, which I've done. This removes the blue cloud icon, but the files are still unselectable within Pythonista.
Edit 2: Renaming the file from "mydata.dat" to "mydata.txt" lets Pythonista import the file, so it seems like this is some kind of file extension issue. It's a bit cumbersome (and in some cases not an option) to have to rename files. Is there a way around this/is this intended behavior?
Additionally, importing the files from dropbox does not allow the np.loadtxt("./mydata.txt") to be found - I have to actually copy the files in to pythonista for this to work. Is there information on the proper path formatting to find a file in Dropbox?
Am I doing something silly here? Is this a dropbox limitation and would be resolved if I used iCloud? (Edit: Issue remains when using iCloud) Any help would be appreciated!
dgelessus last edited by dgelessus
I can't test with Dropbox, as I haven't updated to iOS 11 yet, and Dropbox doesn't provide a document picker for older iOS versions. So I tested a bit with iCloud Drive instead. I made a new folder and used another app to create two files in it: a
data.txtcontaining some text, and a
code.pywith some simple code that reads from
data.txt. When I opened the
code.pyin Pythonista and ran it, I got a permission error when it tried to open the
data.txt. This indicates that a file with that name exists, but Pythonista isn't allowed to access it. After I also opened the
data.txtin Pythonista, the permission error went away and the script worked as expected.
I'm guessing that something different is happening with the Dropbox app. If I go into the Dropbox app on iOS 10, "export" a file, select Pythonista's share extension, and print out the file path, it seems to be placed in a folder with a random name created on the fly - the path doesn't even stay the same if I export the same file multiple times.
I don't know if the behavior with the document picker on iOS 11 is the same, but if it is that would explain why the script can't find the data file - the two aren't actually in the same folder if they are opened from the Dropbox app. To test this, you could print out the paths of the Dropbox files when you open them in Pythonista. To get the path of the file in the current editor tab, run
import editor; editor.get_path()in the Python console. If you do that for your script and your data file (after opening each one from Dropbox), you can compare the paths and see if they are actually in the same folder or not.
@dgelessus - sure enough, it looks like files imported from Dropbox are put in to different, random, folders on the filesystem for each imported file, though I did not check to see what happens if files are batch imported. Which is fair enough. Just means that I need to import files in to Pythonista before working with them. A bit clunky on Apple's part, but no big deal. I can work with that.
Now I'm just trying to figure out if the file import dialog only allows specific filetypes to be selected by design, or if this is something that can be fixed by the developer. Any idea there? My concern is that things like images, etc, might not be importable which is kind of important.
I'm not sure if this is Apple's fault - after all with iCloud Drive it seems to work fine, so apparently there is a way to do it without breaking the folder structure. However that could also be Apple special-casing its own apps as they like to do.
Also it's apparently very difficult to write a working document picker, because for all apps I've tried to open files from (Documents by Readdle, OneDrive, and USB Disk Pro) nothing happens after I select a file in the document picker. It's not a Pythonista issue, this happens when using the document picker from other apps as well.
The restricted file types in "Open..." and "Import..." is a known issue. For importing files, as a workaround you can share the file you want to import, select Pythonista's share extension ("Run Pythonista Script"), then select "Import File" in the menu. This works for all file types.
@dgelessus Thank you! Your tip of 'sharing' my file out of the files app got everything fixed up and running for me. I can get the data in, process it, and send the information back out to dropbox in the form of a plot or dat file. Hopefully a solution will come along before long that will allow the files to be operated on directly in dropbox so that they don't need to be copied, but for now this will definitely work.
I'd honestly like to like iCloud Drive - $1/mo for 50 GB would work great for me since I don't have enough data to justify the $10/mo for Dropbox Pro. My only problem is that some games, etc, like to put files in to the Documents folder and there seems to be no way to exclude those from the backup. Hopefully some day this changes.
You can turn off syncing the Desktop and Documents folders in the iCloud preferences on your Mac. That way you can manually move things in and out of iCloud Drive to decide what gets synced, like with Dropbox.
If you want to use the Desktop/Documents sync feature, but exclude certain folders, Apple doesn't provide a good way to do that. You can add
.nosyncto the end of a folder name to exclude it from iCloud Drive sync, but that obviously won't help when the paths are hardcoded into the application...
You could try moving the game data folder to some location that's not in Desktop or Documents, and create a symlink in Documents that points to the new location. This need to be done in the Terminal using
ln -s path_to_the_link_target name_of_the_symlink, because the Finder only lets you create aliases (which are different from symlinks, even though the Finder displays symlinks just like aliases). The game will automatically follow the link and find its files, but iCloud Drive will only sync the symlink, and not the files that it points to. (Actually, I haven't tried putting a symlink in iCloud Drive yet - maybe it won't get synced at all, but that would be fine here too.)
Also, another possible solution I thought of to use data files from Dropbox in Pythonista: you can "open" the data file in Pythonista, use
editor.get_path()to get its full path, and then hardcode that into your script. As far as I can tell, once Pythonista has "opened" a file from Dropbox, it is allowed to access it forever, even after you close the editor tab for the file. Of course this is far from optimal, it requires modifying every script to use the full paths, and it only works if the data file has a file extension that Pythonista accepts.
cpyth last edited by
I know I’m way late, but try either the os or the pathlib module to create a symlink, this makes it so you can edit files in a local directory but still have them sync with the files app.