Nic :@ Henao I don’t know of anything that works on arm currently (mobigen or KindleGen). Might be a good opportunity to open a suggestion with Kindle’s support.

Unfortunately Calibre is the only program I’ve found that generates .mobi files on ARM, which is a shame because Calibre really stresses it. I’ve been using Calibre on a Pogoplug running Debian and as long as you give it a fair amount of time to work it’ll absolutely feed a Kindle.

Looks like the design was changed. economist.html only got mbp:pagebreaks and not article contents

Today I updated the version of the script I’m hosting here:
http://nicj.net/files/economist_to_kindle.phps

It’s based off of CrossCode’s updates, but disables part of the script (NCX TOC generation) that I can’t get working with today’s economist.com and mobigen_linux.

Let me know if it works for you or not.

First off, thanks for this script! I’m looking forward to getting it to work, I have set up everything as instructed but only seem to get the index page on my kindle

am I missing something?

Thanks again
Ian

I haven’t personally tried this on Windows, but others that have commented here have. Good luck!

You use Windows, /usr/local/bin is Linux 😉

The reason why it did not work for me was simple: I provided the wrong baseDir path – make sure that you provide the WHOLE filepath, not only the one which is accessable by the webdirectory!

Peter

thanks for the great script!

But unfortunately it does not work, I encounter the same issue as Gerardo with “Could not log in to economist.com: Username or password mismatch!”.

I reckon it has something to do with cookie handling, but setting all rights to 755 and changing line 90

from

$GLOBALS[‘cookieJarFile’] = $baseDir . ‘/economist-cookies.txt’;

to

$GLOBALS[‘cookieJarFile’] = $baseDir . ‘economist-cookies.txt’;

didnt work either.

Any hints?

Thanks,
Peter

Changes
* better handling of images (afaik it doesn’t miss any)
* it’ll optionally read its configuration from economist_to_kindle.config.php
* print the name of the articles as it downloads them (mostly for debugging)
* optionally tell the kindle to treat the article as a magazine (since not all the kindle magazine features are supported thanks to a complete lack of documentation, some people may not like this option much)
* downloads the issue cover (although it can be quite difficult to get the kindle to show it to you, and it’s VERY low resolution)
* verified to work with back issues, although issues before June of 2010 use special compatibility code 🙁 (not a problem for using the script, just a problem when editing it)

Known Bugs
* any kindle menu with “section” in the name (fex. “Go to Section…”) will cause the kindle to close the book with an errork. There’s no permanent damage from this.
* I’ve only tested this with kindlegen 1.1 on linux and kindle software v2.5.2. YMMV.

Download:
http://www.crosscode.org/public/economist_to_kindle.phps

Sample economist_to_kindle.config.php:

It looks like The Economist website has changed again. I’ve made some modifications to the script so it works with the latest print issue from the website.

This is the first time I’m using this script, so I didn’t have the old HTML files for comparison. I also don’t have access to the online-only subscription, and haven’t tested any back-issues. Therefore you might have problems using my modified script for these things — however it works fine for the latest print issue available from the website.

I changed the login to use HTTPS instead of HTTP, so your password is sent encrypted (hopefully — I didn’t veryify this). I also added a few more options, including an option to disable Email sending, and an option to enable creating a link to the latest issue. The link creation allows you to have a single URL on your webserver which will always provide you the latest issue (which you can then download using your Kindle webbrowser).

Here is the script:
https://www.revlogic.net/public/economist_to_kindle.phps
(hit “Accept” / “Allow” / “Add Exception” / etc.. if you get a warning about SSL certificates while downloading the script)

You’ll also need mobigen_linux, which can be downloaded from here:
http://nicj.net/files/mobigen_linux.tar.gz
Or alternatively you can use kindlegen instead of mobigen_linux (though I haven’t tried):
http://www.amazon.com/gp/feature.html?ie=UTF8&docId=1000234621

Feel free to send me any comments or provide fixes for any broken features that I didn’t test.

-slifox

397c397
$endPos = getArticleEnd($article);
414c414
$endPos = getArticleEnd($article);
520c520
function getArticleEnd($article)
539,543d538
< $end4 = strpos($article, '<script', $startPos);
< if ($end4) {
< $ends[] = $end4;
< }
<

Unfortunately, I’m having trouble with back articles. Parser seems thrown by the article subscription links (I’m an online only subscriber). Seems like it would be easy to fix, but the php is throwing me. Halfway tempted to rewrite in ruby, but I’m sure that it’s not worth it, since the script works great for current artciles.

Thanks for finding this issue. I’ve updated the source with 3 fixes:

1) As you found, many of the articles’ links didn’t work and the entire article was missing. This seemed to hit the “Americas” section a lot. I found that if you go to the URL linked from the This Week’s Edition page, the server 301 redirects to a second URL. CURL, which downloads the articles, wasn’t setup to follow this redirect, so those links were missing.

2) Starting with this week, 01/23/2010, the HTML format of many articles changed, which broke the converter (there was an extra H1 in the HTML, which we thought was the title). This caused extraneous SCRIPT tags to be in the HTML, which caused mobigen to break completely. This should be fixed.

3) I’ve updated the file to make sure all “See article” links work (for example, in the Politics This Week section). Before, you would just see the text “See article”. Now, it should link to the actual article.

Let me know of any problems.

http://nicj.net/files/economist_to_kindle.phps

“Another Peter” … sounds like you’re not running it from the command line. My guess is that you have the working directory with insuffucient slashes “/” or something, or maybe you’re missing the curl piece? Either way, you shouldn’t just “double click” on the php file… Try to run it from within the command line window to begin with, and post the output here.

Some of your users (especially Linux/Mac) may want to know that kindlegen has more or less replaced mobigen_linux. A quick update of this line works (replace with kindlegen– all other options work the same, it seems.)
system(“/usr/local/bin/mobigen_linux -c1 $opfFile”);

I’m not sure what it is exactly, but it seems like many links don’t work off of mbp_toc.html. I’ve traced it back to the correct formatting in the economist.html file. When there’s a good , then it works, but half the time it’s just that gets printed out. Is there something whacked here… maybe a substitution or other that’s wiping out the id? Without it, I can’t really use the table of contents.

Also, the NYT allows you to click the ‘trackball’ to the right, and jump to the next article. I imagine that’s done by inserting a tag in the html. Would be great to have that too.

Thanks!

You should be able to just update the basedir with the directory you’ve specified. That path should work on Windows. Since you’re using forward slashes (\), you’ll have to escape them:
C:\\Documents and Settings\\LHY\\My Documents\\Economist

Hopefully that’ll work!

Could you also tell me how to set the basedir to store the files on the directory C:\Documents and Settings\LHY\My Documents\Economist? Thank you.

I added
//set proxy stuff
curl_setopt($ch,CURLOPT_PROXY,”webproxy:8080″);
curl_setopt($ch,CURLOPT_PROXYUSERPWD,”userdomain\\username:password”);

twice (in function economistLogin() and function economistGetUrl($url))
Thanks for your feedback!