23. August 2016

Bad Design, Finalising & a Lookback

Hi, since this is the final one, and because I’m somebody who always has their Eureka moments in the nick of time, this is going to be a long one. But I think you’ve had enough of short and sweet from my side.

If you don’t like to read much, I suggest you turn back now.


07. August 2016

Code Review, Optimizations and Formal Benchmarking


Firstly, sorry for the extremely late blog post this time around, however I was waiting on my mentor’s comments before I gave another status report because I wanted his take on where we are progress wise and the status of the pull request. So without further ado, let’s get into it.


08. July 2016

Benchmarking and Unit Testing

Hi, sorry this post a couple days later than warranted, but there’s been some work that required being taken care of. Since the end of the midterm evaluations, efforts were concentrated into taking care of issues such as compatability with the older API of scrapy, unit testing and benchmarking the signal performance with the new API and looking for places where further optimizations are required.


24. June 2016

Midterm Evaluations

What’s done

In the time that has passed since I last posted one of these, I managed to get a prototype of scrapy to work using the new Signals API. This introduced two very significant API changes into Scrapy.