taboca/challenges-python

Open bugs and challenges related to a private repository

challenges-python's Issues

Parsing of e-mail body improvements when no markdown content is provided

class handlingAnonymousIncomingEmail(InboundMailHandler):
  def receive(self, mail_message):

        logging.info("Received a mail_message from: " + mail_message.sender)
        logging.info("The email subject: " + mail_message.subject)
        logging.info("The email was addressed to: " + unicode.join(mail_message.to, ', '))

        try:
            logging.info("The email was CC-ed to: " + unicode.join(mail_message.cc, ', '))
        except exceptions.AttributeError :
            logging.info("The email has no CC-ed recipients")

        try:
            logging.info("The email was send on: " + unicode(mail_message.date))
        except exceptions.AttributeError :
            logging.info("The email has no send date specified!!!")

        plaintext_bodies = mail_message.bodies('text/plain')
        html_bodies = mail_message.bodies('text/html')

        title = mail_message.subject.strip()

        for content_type, body in html_bodies:
          decoded_html = body.decode()
          logging.info("content_type: " + content_type)
          logging.info("decoded_html: " + decoded_html)

          plaintext_bodies

        bodyStr = unicode("")

        for b in mail_message.bodies("text/plain"):
          body_type, pl = b
          try:
            if pl.encoding:
              body = pl.payload.decode(pl.payload.encoding)
            else:
              body = pl.payload
            logging.info(u"---->" + body)
            bodyStr = bodyStr + unicode(body)

          except Exception, e:
            logging.debug("Body: %s" % (pl))
            bodyStr = bodyStr + unicode(pl)

        logging.info(u"***" + bodyStr)

        transientUniqueId = transient.fromEmail_transientPresentation_new(mail_message.sender, bodyStr)
        # future...
        #transientUniqueId = fromEmail_transientPresentation_new(mail_message.sender, bodyStr, attachments)
        logging.info('transientPresentation UniqueID response:  ' + transientUniqueId)
        logging.info("-- e-mail received  --")

calls

def aSync_transientPresentation_process(anonymousEmail, anonymousBody):

        logging.info('cron :: transient ->  ')
        # Splitting markdown
        strHTML = mDown(anonymousBody)
        # https://docs.python.org/2/library/xml.dom.minidom.html
        domDoc = parseString('<root>'+strHTML+'</root>')
        slides = domDoc.getElementsByTagName("section")

        logging.info('cron :: transient -> mdown :' + anonymousBody)
        logging.info('cron :: transient -> html :' + strHTML)

        for slide in slides:

            serializedNode = slide.toprettyxml(indent = "", newl='\n', encoding = 'utf-8')
            logging.info("SerializedSection:" + serializedNode)

            #    title = slide.getElementsByTagName("h1")[0]
            #    print "<p>%s</p>" % getText(title.childNodes)
            h = html2text.HTML2Text()
            h.ignore_links = False
            sectionInMarkdown = h.handle('\n'+serializedNode)

            logging.info("Back to markdown:" + sectionInMarkdown)

            bodyStr = '\n\n'+sectionInMarkdown

            logging.info("To formdata:" + bodyStr)

This works if e-mail content has sectionized (#,##,##,..) markdown structures. When the e-mail has just plain content, what we have strHTML = mDown(anonymousBody)
is a e-mail header info.

Cron handler to work within 1 minute

http://stackoverflow.com/questions/15497074/google-app-engine-cron-schedule-and-length

Transient presentations created from e-mail til post-validation

Transient article ecossystem

E_mail

• email
• email data
• e-mail broker: push transientPresentation

Cron-based transaction

• query transientPresentation with process_stage= not_processed. The transientPresentation in fact represents the act of requesting data to become an article, which was initially performed via e-mail. It would make sense to say "transientTransaction" or "pendingTransactionMakePresentation".

Events of processing

• Pull or create an user Contact in its transient state (Contact.signature = 'transient email')
• create Article associated with such Contact
• create Slides and assign to such article
• send validation request e-mail and modifies process_stage=processed_pending_validation

slides-live runtime and supervisor/managers

We have

slides-live.js (working with tv3)
slides-live-fullpage.js (working with fullpage)

This is seed to an architecture so that we can replace the supervisor animation player and/or have multiple of them working.