This is the summary of my work done during Google Summer of Code 2018 with Debian.
Project Title: Wizard/GUI helping new interns/students get started
Final Work Product: https://wiki.debian.org/MinkushJain/WorkProduct
Mentor: Daniel Pocock
Codebase: gsoc-2018-experiments
Google Summer of Code is a global program focused on introducing students to open source software development. Students work on a 3-month programming project with an open source organization during their break from university.
As you can probably guess, there is a high demand for its selection as thousands of students apply for it every year. The program offers students real-world experience to build software along with collaboration with the community and other student developers.
This project aims at developing tools and packages which would simplify the process for new applicants in the open source community to get the required setup. It would consist of a GUI/Wizard with integrated scripts to setup various communication and development tools like PGP and SSH key, DNS, IRC, XMPP, mail filters along with Jekyll blog creation, mailing lists subscription, project planner, searching for developer meet-ups, source code scanner and much more! The project would be free and open source hosted on Salsa (Debian based Gitlab)
I created various scripts and packages for automating tasks and helping a user get started by managing contacts, emails, subscribe to developer’s lists, getting started with Github, IRC and more.
I made a script for fully automating the subscription to various Debian mailing lists. The script also automates its reply process as well to complete the procedure for a user.
It works for all ten important Debian mailing lists for a newcomer like ‘debian-outreach’, ‘debian-announce’, ‘debian-news’, ‘debian-devel-announce’ and more.
I also spent time refactoring the code with my mentors to make it work as a stand-alone script by adding utility functions and fixing the syntax.
The video demo of the script had also been added in my blog.
It inputs the email and automated reply-code received from @lists.debian.org from the user, and subscribes them to the mailing list. The script uses requests library to send data on the website and submit it on their server.
For the application task, I also created a basic GUI for the program using PyQt.
Libraries used:
This is a working demo of the script. The user can enter any Debian mailing lists to subscribe to it. They have to enter the unique code received by email to confirm their subscription:
This task involved writing program to simplify the setup procedure of Thunderbird for a new user.
I made a script which kills the Thunderbird process if it is running and then edits the ‘prefs.js’ configuration file to modify configuration settings of the software.
The program overwrites the existing settings by creating ‘user.js’ with cusotm settings. It gets implemented as soon Thunderbird is re-opened.
Also added the feature to extend the script to all profiles or a specific one which would be user’s choice.
Features:
Examines system process to find if Thunderbird is running in background and kills it.
Searches dynamically in user’s system to find the configuration file’s path.
User can chose which profile should they allow to change.
Modifies the default settings to accomplish the following:
and many more…
Libraries used:
I created a program to analyse user’s project directory to find which Programming Language they are proficient.
The script would help them realise which language and skill they prefer by finding the percentage of each language present.
It scans through all the file extensions like (.py, .java, .cpp) which are stored in a separate file and examines them to display the total number of lines and percentage of each language present in the directory.
The script uses Pygount library to scan all folders for source code files. It uses pygments syntax highlighting package to analyse the source code and can examine any language.
Libraries used:
I added a Python script with all common file extensions included in it.
The script could be excecuted easily by entering the directory’s path by the user.
Research:
Searched Python’s glob library to iterate through home directory.
Using Github Linguists library to analyse code.
Pygments library to search languages through syntax highlighter.
This is a working demo of the script. The user can enter their project’s directory and the script will analyse it to publish the result:
For managing contacts/calendar for a user, Thunderbird extensions need to be installed and setup.
I created a Debian package for CardBook, a Thunderbird add on for managing contact using vCard and CardDAV standards.
I have written a blog here, explaining the entire development process , as well as using tools to make it comply to Debian standards.
Creating a Debian package from scratch, involved a lot of learning from resources and wiki pages.
I created the package using debhelper commands, and included the CardBook extension inside the package. I modified the binary package files like changes, control, rules, copyright for its installation.
I also created a Local Debian Repository for testing the package.
I created four updated versions of the package, which are present in the changelog.
I used Lintian tool to check for bugs, packaging errors and policy violations. I spent some time to remove all the Lintian errors in 1.3.0 version of the package.
I took help from mentors on IRC (#debian-mentors) and mailing lists during the packaging process. Finally, I added mozilla-devscripts to build the package using xul-ext architecture.
I updated the ‘watch’ file to automatically pull tags from upstream.
I mailed Carsten Schoenert, Debian Maintainer of Thunderbird and Lightning package, who helped me a lot along with my mentor, Daniel during the packaging process.
CardBook Debian Package: https://salsa.debian.org/minkush-guest/CardBook/tree/debian-package
Blog: http://minkush.me/cardbook-debian-package/
I created and setup my public and private GPG key using GnuPg and added them on mentors.debian.net.
I signed the package files including ‘.changes’, ‘.dsc’, ‘.deb’ using ‘dpkg-sig’ and ‘debsign’ and then verified them with my keys.
Finally, the package has been uploaded on mentors.debian.net using dput HTTPS method.
Link: https://mentors.debian.net/package/cardbook
This is video demo showing the package’s installation inside Thunderbird. As it can be clearly observed, CardBook was successfully installed as a Thunderbird add-on:
One of most challenging tasks for a new contributor is getting started with Internet Relay Protocol chat and its setup.
I made an IRC Python bot to overcome the initial setup required. The script uses socket programming to connect to freenode server and send data.
Features:
*It registers new nickname for the user on Freenode server by sending user’s credentials to Nickserv. An email is received on successful registration of the nickname.
The script checks if the entered email is invalid or the nickname chosen by the user is already registered on the server. If this is case, the server disconnects and prompts the user again for re-entering the details.
It does identification for the nickname on the server before joining any channel by messaging ‘nickserv’ , if the nick registration is successful.
It displays the list of all available ‘#debian’ channels live on the server with minimum 30 members.
The script connects and joins with any IRC channel entered by the user and displays the live chat occurring on the channel.
Implements ping-pong protocol to keep the server live. This makes sure that the connection is not lost during the operation and simulate human interaction with the server by responding to its pings.
It continuously prints all data received from the server after decoding it with UTF-8 and closes the server after the operation is done.
Libraries:
Socket library
This is a working video demo for the IRC script.
To display one of it features, I have entered my already registered nickname (Mjain) to test it. It analyses server response to ask the user to again enter it.
I created scripts using Selenium Web Driver to automate new account creation on Salsa and Github.
This task would provide a quick-start for a user to get started to contribute to Open source by registering account on web-hosting clients for version control.
I learned Selenium automation techniques in Python to accomplish it. It uses web driver to control it through automated scripts. (Tested with geckodriver for Firefox)
I used Pytest to write test scripts for both the programs which finds whether the account was successfully created or not.
Libraries used:
The aim for this task was to extract data from user’s email for ease of managing contacts.
I created a script to analyse user’s email and extract all Phone numbers present in it. The Program fetches all mails from the server using IMAP and decodes it in using UTF-8 to obtain it in readable format.
Features:
Easy login on mail server through user’s credentials
Obtains the date and time for all mails
Option to iterate through all or unseen mails
Extracts the Sender, Receiver, Subject and body of the email.
It scans the body of each message to look for phone numbers using python-phonenumbers and stores all of them along with details in a text file in external system.
Features:
Converts all the telephone numbers in Standard International Format E164 (adds country code if not already present)
Using geocoder to find the location of the phone numbers
Also extracts the Carrier name and Timezone details for all the phone numbers.
Saves all this data along with sender’s details in a file and also displays it on the terminal.
Libraries used:
The original libphonenumbers is a popular Google’s library for parsing, formatting, and validating international phone numbers.
I also researched Telify Mozilla plugin for a similar algorithm to have click-to-save phone numbers.
This is a working video demo for the script:
I have created another script to automate the process of new account creation on Salsa using HTTP Post.
The script uses requests library to send HTTP requests on the website and send data in forms.
I used Beautiful Soup 4 library to parse and navigate HTML and XML data inside the URL and get tokens and form fields within the website.
The script checks for password mismatch and duplicate usernames and creates a new account instantly.
Libraries used:
This is a working demo for the script. An email is received from Salsa which confirms that new account has been created:
One of the problems faced by a developer is filtering hundreds of unnecessary mails incoming from mailing lists, promotion websites, and spam.
Email client does the job to certain extent, still many emails are left which need to be sorted into categories.
For this purpose, I created a script which examines user’s mailbox and filters mails into labels and folders in Gmail, by creating them. The script uses IMAP to fetch mails from the server.
Libraries used:
I would like to thank Debian and Google for giving me this opportunity to work on this project.
I am grateful to my mentors Daniel Pocock, Urvika Gola, Jaminy Prabharan and Sanyam Khurana for their constant help throughout GSoC.
Finally, this journey wouldn’t have been possible without my friends and family who supported me.
Special Mention
I would like to thank Carsten Schönert and Andrey Rahmatullin for their help with Debian packaging.