Python script for Subscription to Mailing List

I am applying for Google Summer of Code this year under Debian. I got to know about this programme through my brother, who was selected for GSOC to contribute to a Debian project last year. He really inspired me to apply to GSOC, even though I hate to admit this publicly :P GSoC is a great opportunity for students like me and many others, to start contributing to open source softwares under the guidance of mentors.

This motivated me to give my best and apply this year. I decided to work on the project Wizard/GUI helping students get started under Debian. It aims at developing a tools and packages which would simplify the process for new applicants in the open source community to get the required setup. For example, installing useful development tools on their computer, subscribing to relevant mailing lists. This project specially interests me as I feel it overcomes the challenges I faced when starting with open source contribution and setting up the environment.

To make a meaningful contribution towards this project, I decided to create a concise Python script. This script automates the process of subscribing to relevant Debian mailing list by sending your email and then confirming it through a customised reply by mail. The input would be entered by the user in a PyQt Based GUI application. Initially, I wasn’t aware of technologies such as PyQt, but I overcame this barrier when I started working on it. For getting started with GSoC, I would highly recommend reading the article by Daniel Pocock (Debian mentor), since it personally helped me.

The blog explains how one could start contributing to the open source community by creating a github account, making a blog, joining relevant IRC channels and mailing lists etc.

GIT

Git is the fundamental requirement for collaborating on open source projects. It’s a “Distributed Version control” which allows people to work on same project and share the code and everyone knows what changes you and others have made.

Initially I was not well versed with Git but during this application period, I learned important concepts by watching youtube videos and reading blogs. It helped me Learn the basic concepts such as repositories, git command prompt, forking, pull requests etc. I also completed a quick 15 minute course here for learning basic git commands. While learning Git, all the commands and concepts may seem confusing and tedious, but it comes naturally when you start using it for projects.

PROJECT

The initial contribution I made for the project can be divided into 3 parts for better understanding:

• Submitting user email on the Debain-outreach website website
• Confirming Subscription through a mail reply
• Making a Python GUI for the above tasks

We would be able to do exactly like the video shown below. Sounds interesting?

Submitting user email on the website

To subscribe to the mailing list, I have to enter my email on this website and click on the subscribe button through a python script. On successful implementation, the user should get a mail confirming the same.

This is where the users's email will get entered.

First, I learned about Python scripts that can be used to automate this process. It involves using python "requests" library which can send custom request (Post/Get) to a webpage. We don’t have to manually add query-strings to URLs or encode your POST data. You can easily add headers and pass data using this library. It has pre-configured functions that help you do that easily.

Another method that can be used to automate tasks is by using “Selenium Webdriver”. It provides a open source API for automating actions on website and can be integrated in your internet browser. But, the major drawback of using it is that it is too slow as compared to importing requests library in python. We could have also used urllib2 library but it adds more complexity.

The ‘requests’ library can be imported into python through command prompt by two ways:

• $ pipenv install requests : This command uses “pipenv” which is a ‘dependency manager’ for python. It’s a high level tool which essentially simplifies the process of installing libraries.
  Use pip to install Pipenv: pip install --user pipenv


• $ pip install requests : ‘pip’ is the default method used to install software packages written in python

I personally used the 2nd method as it is straightforward and installs basic packages without any hassles.

Also, I installed Git Bash, which allows to run Unix commands and also do additional tasks like search command history, make user-defined functions etc.

References:
Installation of Requests
Virtual environment

For learning about its functioning , I wrote a code to automate the process for logging into Github account.

import requests
with requests.Session() as c:
    
    url="https://github.com/login"
    email=input("Enter you email: ")
    PASSWORD=input("Enter your password: ")
    c.get(url)
    login_data=dict(username=email, password=PASSWORD)
    c.post(url,data=login_data)
    page=c.get("https://github.com/minkushjain?tab=repositories")
    print(page.content)

The c.get pings a website for information at that specific url (making request). The web server responds and you can collect the content using the ‘get’ function. This could successfully login into my github account. I also tried some other tasks like downloading data (media) from different websites.

Next, for the original task, we had to enter your email in this link.

CODE:

 
import requests
with requests.Session() as c:
    url="https://lists.debian.org/cgi-bin/subscribe.pl"
    email=input("Enter you email: ")
    c.get(url)
    login_data={'user_email':email, 'list':'debian-outreach', 'action':'Subscribe'}
    c.post(url,data=login_data,stream=True)

The email data had to be entered into the https://lists.debian.org/cgi-bin/subscribe.pl link as it can be discovered from the source page of the Mailing list web page.

Source code of the link highlighting the link.

After importing requests module, it asks the user to enter his email. Requests allow us to enter the data for the url as a dictionary of strings, which has been stored in login_data.

Now, we have Response object called ‘c’ using which we can get all the information.
eg. If you want to enter key1=value1 and key2=value2 into httpbin.org/get , then you can store these values in a dictionary named ‘user_data’ and then enter it by:

user_data = {'key1': 'value1', 'key2': 'value2'}
r = requests.get('http://httpbin.org/get', params=user_data)

This is how you can enter data through post method of HTTPS on a web page.

c.post(url,data=login_data,stream=True)

I learned that HTTP has 2 types for inputting data: POST and GET

• ‘Get’ request includes all the user data into the URL. These parameters are shown in the URL and hence get stored in history
• ‘Post’ request submits the user data into a specific resource instead of showing up in the URL, hence it is a more secure way.

The Debian mailing list uses POST method since the login data doesn't get shown in the url of the website.

Confirming Subscription through a mail reply

After the clicking the subscribe button in the previous step, the user receives a mail. The user has to reply to that mail with Subject String as specified. To simplify this purpose, I learned about SMTP protocol (Simple Mail Transfer Protocol)

This is the basic code for sending a mail from one email to another using SMTP Protocol. It is basically a IP protocol used in sending and receiving e-mail.

import smtplib
server=smtplib.SMTP("smtp.gmail.com",587)
server.starttls()
server.login("your_email","password")
msg="Hello...sending message from smtp server protocol"
server.sendmail("your_email","receiver_mail",msg)
server.quit()

The server.login syntax intakes your username and password and logins into your mail.

from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText

This is a must for sending a mail with customized subject.
The ‘587’ is the port through which the email is being sent. You can use other ports as well but every port leads to a different output!

I got multiple errors in the first code, stating that the authorization failed and it couldn’t connect to the network.
To fix these, I allowed “installation of less secure 3rd party apps” in gmail settings. The statement “server.starttls()” also has to be added before authorisation into your account to get it working.

CODE:

import smtplib
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
a=input("Enter the code of the subject you received in your mail: ")
b="CONFIRM "+a
fromaddr="sender_email"
toaddr="debian-outreach-request@lists.debian.org"
msg = MIMEMultipart()
msg['From'] = fromaddr
msg['To'] = toaddr
msg['Subject'] = b
 
body = "Subscribe"
msg.attach(MIMEText(body, 'plain'))
 
server = smtplib.SMTP('smtp.gmail.com', 587)
server.starttls()
server.login(fromaddr, "sender_password")
text = msg.as_string()
server.sendmail(fromaddr, toaddr, text)
server.quit()

The mail could be sent from any email id regardless of user’s email. After entering the Subject string in the mail, it composes a mail to ‘debian-outreach-request@lists.debian.org’ and eventually the user gets subscribed. I have concatenated the user code with 'CONFIRM' string for the subject of the mail.

This is just my first blog, I hope I am able to explain about the work I did. The journey till now has been exciting. In a short span of time I have learned about new meaningful technologies. I am discovering my potential and I am very motivated to move along with my work.

Thank you for reading!!

GUI for the program using PyQt

I will cover how to make UI like this is next post but you can already see the whole code along with UI on my gsoc-debian repository.

---