Copy code from forum

[Phuket2]

This code was for the old forum. It still appears to work. One difference being that it appears to be a paged system here, so for long posts it will not get all the code. I will think about that later. Still wanted to post it here for any feedback/suggestions anyone may have.

A few warnings up the top of the file worth reading....

# coding: utf-8

# WARNING
# If you click write to file, a directory called
# _temp_dir_name will be created in your root dir
# i think the normal temp/tmp dirs are invisible
# in pythonista. i refer to as a temp dir as in 
# my case, i only download code from the forum
# to test it. i want it all to go to one dir. 
# easier to clean up 

# WARNING
# Something wrong with the clipboard code. I have
# hidden the button with a constant below until i
# can workout why i am corrupting the clipboard.
# at least least, i think thats what is happening 
_SHOW_CLIPBOARD_BTN = False 

# if this is set to true, the file created will
# be opened in the editor. According to the docs
# the current file is automatically saved
_OPEN_FILE_AFTER_WRITE = True  

import ui
import clipboard, console, editor
import os, time
import tempfile
import requests
import bs4


# name of the directory that will be used to save
# the code files if you write to disk. if the dir
# does not exist it will be created.
_temp_dir_name = 'forum_temp/'

# the full path of the temp dir, saving here!
__TEMP_DIR__ = os.getenv('HOME')+'/Documents/' +  _temp_dir_name

class OmzFourmCopyUI(ui.View):
	def __init__(self, data_list, post_title):
		#the list of code blocks, scraped from the forum
		self.data = data_list
		self.name = post_title
		
		# create acopy title btn
		# using the var below to hide it. i think
		# i could be corrupting the clipboard somehow
		if _SHOW_CLIPBOARD_BTN:
			btn = ui.ButtonItem('Clipboard')
			btn.action = self.copy_code
			self.left_button_items = [btn]
		
		# create copy to file btn
		btn = ui.ButtonItem('Write to File')
		btn.action = self.write_to_File
		self.right_button_items = [ btn]
		
		# create the tableview, set the data_source
		# and delegate to self.
		tv = ui.TableView(name = 'tv1')
		tv.name = 'tv1'
		tv.data_source = self
		tv.border_width = .5
		tv.delegate = self
		self.add_subview(tv)
		
		#create textview
		txtv = ui.TextView(name = 'txtv')
		txtv.font = ('Courier', 14)
		self.add_subview(txtv)
		
		# select the first code block
		tv.selected_row = 0
		txtv.text = self.data[0]

	# copy the selected code block to the clipboard 
	def copy_code(self, sender):
		if self['txtv'].text:
			code = self.data[self['tv1'].selected_row[1]]
			clipboard.set(code)
			console.hud_alert('Copied', duration = .5)
			self.close()
			
	# write the selected code block to a file
	# tempfile is used, delete = False
	# a var at the top of this file determines
	# if the file is opened after its written
	def write_to_File(self, sender):
		if not self['txtv'].text:
			console.hud_alert('No text to write to file', duration = 2)
			return
		
		# create a prefix for the file, using local
		# time of the device
		the_prefix = time.strftime('%X', time.localtime()).replace(':', '_') + '_'
		
		# if no directory exits, create one
		if not os.path.exists(__TEMP_DIR__):
			os.mkdir(__TEMP_DIR__)
		
		# create and return a ref to the new file
		fh = tempfile.NamedTemporaryFile(dir = __TEMP_DIR__, prefix =  the_prefix, suffix = '.py', delete = False)
		
		code = self.data[self['tv1'].selected_row[1]]
		fh.write(code)	# write the code to the file
		fspec = fh.name
		fname = fh.name.split('/')[-1:][0]
		fh.close()
		
		#close the view
		self.close()
		
		console.hide_output()
		
		if _OPEN_FILE_AFTER_WRITE:
			editor.open_file(fspec)
		else:
			console.hud_alert('written to file {}'.format(fname, _temp_dir_name), duration = 2)
		
	# handle the resizing more correctly.		
	def layout(self):
		self.frame = self.bounds
		tv = self['tv1']
		
		tv.frame = (0,0,self.width * .20, self.height)
		
		self['txtv'].frame = (tv.width, 0, self.width - tv.width, self.height)
	
	# tableview data_source callback methods	
	def tableview_cell_for_row(self, tableview, section, row):
		
		cell = ui.TableViewCell()
		cell.text_label.text = 'Code {}'.format(row + 1)
		return cell
		
	def tableview_number_of_rows(self, tableview, section):
		# Return the number of rows in the section
		return len(self.data)
		
	# tableview delegate callback method
	def tableview_did_select(self, tableview, section, row):
		self['txtv'].text = self.data[row]
	

def extract_code_blocks(url): 
	# code from omz as a quickie, i made some mods
	lst = []
	
	html = requests.get(url).text
	soup = bs4.BeautifulSoup(html)
	pre_tags = soup.find_all('pre')
	
	if pre_tags:
		for p in pre_tags:                               
			lst.append(p.get_text())
	
	title = ''
	if len(lst):
		# extract the title of the post
		title = soup.title.get_text()
		title = title.split('—')[-1:][0].lstrip(' ')
		
	# if no code block found, lst will be an empty
	# list and title will be ''
	return (lst, title)
	
	
if __name__ == '__main__':
	url_fail = True 
	console.show_activity()
	# copy a url from the clipboard
	# url should contain a link to a omz forum
	# that you want to extract code from.
	url = clipboard.get()

	#check the url
	if 'forum.omz-software.com' not in url:
		console.alert('omz-forums, not in url')
	elif len(url.splitlines()) > 1:
		console.alert('url, has multiple lines')
	else:
		# if the url looks ok, try and get
		# a list of code blocks from the
		# omz fourm
		url_fail = False 
		result =  extract_code_blocks(url)
		lst , post_title = result
	console.hide_activity()
	
	
	if not url_fail:
		if len(lst) == 0:
			# url appeared ok, but no code blocks found
			console.alert('No code found on page')
		else:
			x = OmzFourmCopyUI(lst, post_title).present()
	else:
		# if the url does not pass the checks
		# print the url to the console
		print 'Clipboard Contents - {}'.format(url)
		

[Phuket2]

Or code in gist I guess.

link text

[Phuket2]

Code on GistHub

Sorry, getting used to things here

[cook]

Maybe try getting your info from the RSS feed of the thread rather than using the URL.

Compare:

(A) https://forum.omz-software.com/topic/2037/copy-code-from-forum
(B) https://forum.omz-software.com/topic/2037.rss

It's easy enough to convert A to B. Then you can load the rss information using feedparser module.

usually something like this...

import feedparser
a = feedparser.parse('url')
for i in a['entries']:
    print i

I don't have time to look at this specific feed - but just wanted to say you may be able to get ALL of the information from a thread using the rss method. Maybe you need to test it on a super long thread.

Easy to load into a dictionary :)

[Phuket2]

@cook. Actually the code itself is ok besides the strange bug. The bug only occurs when you are doing something that is not normal. But of course we have to test for all conditions.
I am a bit quiet now, some friends from overseas arrived. Party party!
Also when I get a chance to do someThing here, I am focusing on github. But it's driving me crazy. It's ok, I will eventually work it out, I hope. Not for the faint hearted!