Stage two: refactoring

5 06 2006

CamembertTo make Cheesecake easier to maintain and more extensible I’ve decided to do some major refactoring this week. You may expect things to be broken inside mk branch for some time. But at the end of the week you’ll hopefully be able to hack your own index and easily plug it into Cheesecake. I will write about it in more detail later. And for all agile fans I will also incorporate automatic creation of documentation and coverage statistics into buildbot.

Oh, and check out what Titus Brown has to say about Cheesecake. If you’re interested in explanation how Cheesecake development process looks like, read last Grig post.

Advertisements




Make your code covered

1 06 2006

I committed a big chunk of code to Cheesecake yesterday, along with a bunch of tests. But how can I be sure I actually covered all of the code? Of course nothing will substitute good suite of tests, but I thought trying out a coverage tool won’t be a bad idea. To install coverage simply download a one-file module and drop it into your PYTHONPATH. You may also think about making an alias for it, so you can easily call it from the command line:

$ alias coverage='python /usr/lib/python2.3/\
site-packages/coverage.py'

I wanted to check what code is executed by current set of tests that Cheesecake have. I’ve used nose runner, with a special option:

$ nosetests --with-coverage

After about 20 seconds it finished its work. Results can be seen by invoking coverage directly. For the Cheesecake package I wanted to check only few modules, because not all are written by us. So the command line went like that:

$ coverage -m -r cheesecake/_util.py\
cheesecake/cheesecake_index.py cheesecake/codeparser.py\
cheesecake/config.py cheesecake/logger.py

Results were as follows (stripped out – irrelevant here – line numbers of not-covered statements):

Name Stmts Exec Cover
cheesecake/cheesecake_index 633 410 64%
cheesecake/config 35 29 82%
cheesecake/_util 47 40 85%
cheesecake/logger 80 70 87%
cheesecake/codeparser 89 87 97%
TOTAL 884 636 71%

As you can see, codeparser code (which I was working on yesterday) is covered quite well. Results also clearly show we have a lot of work to make Cheesecake tested appropriately. 64% is certainly not enough.

HTML table was generated by cover2html.py script.





“The Queen of Cheeses”

29 05 2006

brie.pngReady to taste some of the famous brie? Go on and check out Cheesecake’s first milestone. Along with few bugfixes I plan to enhance the documentation index. I hope to deliver first modifications in next few hours, so feel free to check out my code and test it, before our buildbot does! ;-)





Welcome to the National Cheese Emporium

25 05 2006

Summer of Code is a program sponsored by Google. Participants are paid for development of open-source applications. I’m proud to announce I’ve been chosen by Python Software Fundation to enhance Cheesecake and integrate it with PyPI. You can found out the details in my application and in Grig’s post, who will be my mentor during the SoC. If you care about Python package distribution, please don’t hesitate to check out and fill up our wiki page with your ideas. Source code of my current work will be always available in the mk branch of Cheesecake Subversion repository. We’ll be setting up a buildbot shortly so stay tuned.





SoC 2006 Application

25 05 2006

Project Title

Cheesecake enhancements and its integration with PyPI.

Benefits to the Community

Cheesecake is an application designed to evaluate and estimate the overall quality (or so called ‘kwalitee’) of a given software package written in Python. It emphasizes a need for well-written documentation and unit tests, encouraging good programming practices and penalize sloppy design and careless distribution. Using Cheesecake to check your code gives you confidence that your software doesn’t merely run, but is usable and easy to test and modify as well.

Because Python is very easy to learn and use there exists a vast variety of software written in it, most of which was scattered until PyPI was created. Now, when new packages are being indexed on Cheese Shop every day, effort can be made to spread the spirit of good software design and code reuse among the Python community. This can be achieved by combining the power of Cheesecake and Cheese Shop. Everytime a new version of a package would be uploaded to Cheese Shop, its cheesecake index will be calculated and published on web. Having a way to measure a quality of a package with accordance to other existing packages will be of invaluable help for all developers. It will promote well built packages and in the long run raise the overall quality of Python software.

Details

Adding Cheesecake functionality to PyPI has been already mentioned by Phillip J. Eby on catalog-sig mailing list. Together with Cheesecake maintainer Grig Gheorghiu we’ve discussed modifications needed to be done to Cheesecake code to be reliable enough so it could be incorporated into PyPI service. A working copy of our ideas is accessible on the project wiki. It includes enhancing Cheesecake code scoring techniques to take into account unit tests of a package, running tests in secure environment, extending supported archive formats and fixing all known bugs. Development of Cheesecake will adhere to best practices such as unit testing, continuous integration (via buildbot), pylint verification, etc. This step will take about 5-6 weeks.

The next part of this project will include collaboration with Richard Jones, PyPI maintainer, and merging Cheesecake into PyPI service. Upon completion all PyPI uploads will be automatically scored by Cheesecake. It will be possible to browse packages archive by cheesecake index, sorting results by installability, documentation and code kwalitee index. Statistics in numeric and graphical form will also be made available. This part of a project, involving writing server-side code, with emphasis on security and robustness, will take as well 5-6 weeks.

The remaining time will be spent on resolving all problems that would occur during usage of Cheesecake and PyPI. Along with fixing bugs, I will develop a simple Hello world package that can be taken as an example of good development practices for all Python developers. It should also score 100% in Cheesecake test of course. ;-) It will be what hello is for GNU Project. Implementation shouldn’t take more than two weeks, so I’ll hopefully complete the whole project before the deadline.

UPDATE: Testing of packages

Testing of package code will be performed by running its own tests, either by setup.py test target or by piping each module through doctest. Making unittest-based tests work could be a bit tricky, as we don’t know if running a module will execute an application or its tests, but some common cases could be probably handled as well. Points will be granted for test coverage (proportion of number of code/functions and tests) and their positive results. Package will earn bonus points for adhering to standard setup.py test target, as it’s a preferred method for all setuptools packages.

Packages should be tested often, not only when a new version of given package is published, but also when there are new releases of modules this package require. On the other hand, there’s no need to test changes in not-related packages.

To implement such a system a secure environment must be created, so that malicious tests won’t harm the whole system. Since python doesn’t have restricted execution capabilities since 2.3, some other means of protection would need to be used. I just haven’t decided yet, but will try to keep it as simple as possible. If needed, testing can be moved to another box. Cheese Shop will use a web interface to request testing of certain packages from this test-server and testing software on that box will run all tests and publish results in a fixed location. Although creating such a buildbot may seem difficult, I think it’s possible and can be done in the timeline I’ve presented. I have suitable unix experience as I’ve done some administrative work during my years spent on learning unix systems. Building a web interface is also not a problem, especially in Python. Problems will certainly arise, but that’s what make programming so exciting. I think hard work will do the job and project will be completed before the deadline.

Bio

My name is Michał Kwiatkowski and I study Computer Science at Gdansk University of Technology, Poland. My first programming experience date back to times I was about 9 years old and was coding simple applications in BASIC on C=64. I’ve got seriously involved into programming during my high school years. Since then I learned C, Perl, assembly, Python, and a bit of JavaScript, Ruby and Lisp. Now I’m in a process of learning Scheme from the excellent SICP. During these years of learning and experimentation I’ve managed to write few open source applications. All of them can be downloaded from my homepage. What I like most in programming is discovering new simpler methods of making practical things. I often reimplement a given idea just to find the perfect solution. I’m really interested in programming languages design and how they affect our minds in process of creating the software. I consider Python my language of choice, so I will be very excited to work on a project that will help Python community.

Mentors

I’ve already contacted Grig Gheorghiu and he expressed his willingness to mentor my project. Richard Jones will also be able to co-mentor my efforts to integrate Cheeseshop into PyPI.

Contact information

Name: Michał Kwiatkowski
Email: ruby@joker.linuxstuff.pl
Jabber address: ruby@chrome.pl
Homepage: http://joker.linuxstuff.pl/