############################################################################### # AmphetaDesk (c) 2000-2004 Disobey # # TODO http://www.disobey.com/amphetadesk/ # ############################################################################### ------------------------------------------------------------------------------ SLATED FOR v0.94 RELEASE (ie. try to get these done before everything else) ------------------------------------------------------------------------------ --------------------------------------------------------------------------- Goldplating: --------------------------------------------------------------------------- - look into new channels for use in the defaults. --------------------------------------------------------------------------- New filter_items subroutine: --------------------------------------------------------------------------- - CHANNEL LINKS: Check if the links in a channel's rss are relative ("/relative/click.html") and turn them into $channel->{htmlurl} . "/relative/click.html" (absolute versions). We're gonna move this to a "filter_feeds" routine for the templates, so this bugfix is deferred. - HASHED TITLE AND DESCRIPTIONS: check for HASH again. - SHOW ONLY X ITEMS FROM CHANNEL? No HOWTO would be needed. --------------------------------------------------------------------------- Known bugs: --------------------------------------------------------------------------- - the END {} sucks - needs to check on whether they were loaded. - we don't check for errors on whether the mirror/filesave actually works, causing fun nonesucheries on permission errs, etc. i bet we could make the mirror code more readable in the first place. - fix up the logging messages for downloading. they're confusing. --------------------------------------------------------------------------- Crappy code that needs to be fixed/I'm not satisfied with: --------------------------------------------------------------------------- - I don't like how AmphetaDesk::Channels::load_channel will accept a filename or a string - that seems kludgely to me. at most, we should accept a filename or a URL. support for strings was added to handle AmphetaDesk::Channels::add_url, but that needs redoing too. all of this would be rewritten/revamped when we did our AmphetaDesk::Parser. (merge Channels and MyChannels into one routine. much smarter). - the add_url routine is a mess, going all over the place. it really should be rewritten to handle HTML autodiscovery, subscription regardless of parsability, etc. the current code also doesn't support multiple subscriptions where one of the URLs is https://. - I don't like the AmphetaDesk::MyChannels::load_my_channels code, especially the odd kludge to check for raw data or a filepath. I'm not sure this is really something I can fix without totally overhauling the import/add_url code (which will happen anyways). - I don't really like the default subscription lists being in MyChannels.pm anymore. Or the fact that our dates are bogus defaults. Should we move these back to an external file and use our "import" to bring them in as the user's list? - do the save/load_my_channels/settings really need to accept $passed_path? there's really no time the defaults aren't used. - the writing of the OPML file in XML::Simple sucks, since a value has to be "" not undef, else XML::Simple tries to set the undef as a new hash element. this should get better if we can write our own printer when we switch to Ampheta::Parser. NOTE: We're gonna have to rebuild the OS X gui for the next version of AmphetaDesk, because we're using a new version of expat to fix the UTF8-BOM segfault. We don't need a new binary for Win32, since we "fix" the UTF8-BOM bug by cheating in our load_channel code. ------------------------------------------------------------------------------ SOMETIME SOON (no specific date or release. just nice to get done). ------------------------------------------------------------------------------ - BACKUPS: We do no backups of our important files (myChannels.opml, mySettings.xml, and the log file). We should add a directory beneath /data/ called 'backups' and create weekly backups there (4 weeks worth?). I've got code for this already (check the nhpr backup scripts you wrote). - CGI SUPPORT: $os would become Internet, $connection would be STDOUT? Lots of people want this. The biggest addition would be security settings and user accounts. How do you handle $HOME directories? - CHANNEL DBMS: In future versions of AmphetaDesk, our channel information should be stored in DBM files. This will supersede anything we do with channel meta items (and the matching XML). The primary benefit (above and beyond the channel meta items, which allow "new items in a feed", "show only items newer than Y days or hide items older than X" days) is that we'll be able to keep a local store of RSS data, allowing people to create aggregate feeds on keyword, search through months worth of data, and so forth. - CHANNEL DUPLICATES: Check for duplicate code fails on query strings and 'www' or no 'www'. Query strings we can't really take care of. - CHANNEL META ITEMS: This is all in Morbus' head, but has been discussed on the mailing lists, in #disobey, and was partially implemented in deus' AmphetaOutlines. It requires Digest::MD5. (jack@novagate.com, huftis@bigfoot.com, bringmetruth1@netscape.net, adam@kalsey.com, davew@dsl.telocity.com, iain@******.net, oneiros@dcr.net, johnseq@pobox.com jw@abprosys.com, KlausRusch@atmedia.net, huftis@bigfoot.com, mbrow28@sph.emory.edu pkelly@toronto.cbc.ca, scott@randomchaos.com, thomas@stderr.net, chapman@lambuth.edu ashiel@sportsinteraction.com) - CHANNEL GROUPING / ORDERING: There needs to be some way of ordering the channels. The current plan is to allow channels to be grouped under different headings like "Tech News", "Health News" and so on. Categories can be sorted differently than the channels within. See http://sourceforge.net/mailarchive/message.php?msg_id=1726994. (craw@visi.com, peters@earlham.edu, iain@******.net, jw@abprosys.com, meryl@meryl.net, scott@randomchaos.com, kerim.mail@oxus.net) - CHANNEL HTML: Strip all HTML? Allow only these tags? Problem tags include title, script, and textarea. Tables are relatively dangerous as well, but should be remotely restricted to their own channel. Tags to allow: a h* strong em i blockquote ol ul li img p br? - CHANNEL IGNORING: Some people would like to remember channels that have piqued their interest, but don't want to actually see their items in the channel listing until they call for it (they'd still like to stay subscribed). The idea would be to have the title of the channel shown at the end of the listing, with the number of new items ("Wired News - 7 new items since last viewed"). People would be able to click on the title to see the items that have been hidden from view. This would be possible with categories. (peters@earlham.edu) - CHANNEL LISTS: A user should be able to specify which language they are interested in, and get a list of feeds back to them. This will be pretty easy once the Syndic8 export is happening, although we'll probably have to move to the OCS exports instead (which, in the long run, should give us more power anyways). (sasw@Swernofsky.com) - CHANNEL LISTS: Reenact downloading of updated channel lists. This will require a script on disobey.com that will suck down the data and gzip, and then in AmphetaDesk, it'll have to Compress::Zlib to extract the data to the right location. - CHANNEL LISTS: Reenact "are you subscribed?" check for the lists. Not entirely sure what the visual stimulus should be, but a check box probably isn't it (since that'd cause a 'duplicate channel' error upon submittal). Perhaps the absence of a checkbox and something that says "you're currently subscribed, click here to unsubscribe"? - CHANNEL LISTS: Should be searchable. Standard AND/OR/NOT and URL specific searches should be possible. Can we just hook into the Syndic8 API? Not so sure I like the co-dependance factor. The sad thing is, this isn't a perfect solution since the data available from the feeds aren't adequately described, making bad search results. Whatever we do, we'll have to include some sort of warning about bad data. (richardevanslee@yahoo.com, iain@******.net, jw@abprosys.com, scottdavey@telocity.com) - CHANNEL LISTS: Sorting needs to be smarter, by not alphabetizing by "A Channel Name" or "The Channel Name" and instead just "Channel Name". (oneiros@dcr.net) - CHANNEL SORTING: On the main channels page, a user should be able to sort alphabetically, by date downloaded or added, etc. This is pretty easy due to our get_my_sorted_channels routine. We'll have to include a default setting under My Settings. - CHANNEL UPDATES: Force update of all channels. This needs to be GUI capable for Windows, and also accessible from the web pages. Should we add the "Refresh Channel List" to the web pages as well? How will we stop people from force-refreshing multiple times? Perhaps only allow force refreshes of one channel at a time? Perhaps we implement this as a query string like ?update_channels (meryl@meryl.net). - CGI: Not entirely sure how we're going to handle this, but CGI.pm is anxious to convert URL parameters into parameters for our own usage. This starts causing problems when people are trying to ?add_url, and the URL in question has CGI parameters as well. As of right now, we're telling people to decimal encode those parameters, but not everyone can. - COMMAND LINE FLAGS: -nobrowser, -port, -quiet? - CONTENT-TYPES: We send out text/html for a content-type on any non-image request to our webserver. This should get smarter and default to text/plain for everything but /?html?/. - CUSTOMIZABLE EXPORTS: The collected data of all the downloaded RSS files should be exportable to a template based file, or to a webserver, or even to email. This all needs to be customizable, as well as having a pretty happy GUI. (chris@lockergnome.com, djwudi@yahoo.com) - DOCS: add faq? "how to find your proxy server", "common zip problems". - DOCS: convert the various AmphetaModules into POD so that they can be converted into HTML documentation for the webpages. easier for me to refer developers to them. - DOCS: finish template documentation (add data available for use within the templates, how to use unknown data from RSS feeds, and so on and so forth). - DOCS: finish up the rest of the new v0.93 developer documentation. - DOCS: get deus to write a bit about the "to create a mac os x binary" on the build page (how are icons built? can he blurb a bit about the shell files, and how those are run?) - DOCS: write up HOWTOs for "only xxx letters", "show last x channels", "local weather", "using LWP::Simple to get data from other sites", etc. - ETAGS: From http headers, these are unique id's that change only when the content has changed. Here's an example (the collection of these tags would live in the channel DBM file). While we're adding this, we should throw in Conditional Get support as well (reevaluate our HEAD, too). http://fishbowl.pastiche.org/archives/001132.html - EXPAT: Currently, we strip all BOM from UTF-8 feeds in load_channel. This is because some versions of expat can crash if fed one of these feeds. This isn't much of a problem under Windows or MacOS/OSX, since we control the version of expat shipped. However, we need to detect what version of expat is installed on Linux systems, and if we can't, strip BOM stuff. - GOLDPLATING: Our note() and error() routines should join() the entire @_ so that we don't have to use concat tricks to send out long error messages. Probably gigamiliseconds of savings. - GUI (Mac): Remove the MacPerl menu's. Stick with our own. - GUI (Mac): open_url should hook into MacPerl::Internet (see Fry email). - GUI (Win): Allow an option to shut off the GUI for Windows? - GUI (Win): Minimize on start, minimize on X? (jw@abprosys.com) - GUI (Win): Good points. I'll certainly add the "last time checked" sort of thing to the Systray, but the "display the log file" is a bit stickier. I may just run a shell sort of command ("open /data/AmphetaDesk.log") and have Windows open the program (or the "Open With..." window) that is defaulted to handle .log extensions. Also, change tray icon to reflect downloading status? (jw@abprosys.com) - HTTP AUTHENTICATION: Allow auth to protected Yahoo! Groups and generically, any site that requires a username and password. (marcus@wordit.com, KlausRusch@atmedia.net, mkrus@newsisfree.com) - INSTALLATION: This is primarily important for Linux users and whenever we do CGI based installs, but the wrapper script should show what modules are not installed, and give some instructions on how to download them (either for the system, or into our AmphetaDesk lib/ directory). IOW, we shouldn't fatally die impolitely anymore, but offer users a chance to reconcile. We also need to check for version numbers - we use Vars for CGI.pm, which is relatively new enough that some don't have the correct module version. - LOGGING: We should use ISO dates and times on our output, especially since the logfiles now stick around for 250k. - MULTITHREADED EVERYTHING: The webserver should be multi-threaded, the download code should be multi-threaded, and if we could get the GUI loaded in it's own process, that'd be insanely great. Whether this is all magically possible on every system has yet to be tested. More than likely, this feature won't be possible until we can see if threads work under Classic Mac support, since it doesn't support forks(). I really think this is a cross-platform pipedream. - MULTI-USERS: Make AmphetaDesk support multiple users with one install. There's a whole crapload of email on this. (gordon@g2meyer.com, see also SF feature request) - MY CHANNELS OPML UPDATES: If we receive a 301 HTTP Redirect, we should update the URL in the myChannels.opml (we've got initial code from Klaus to do this). - OPEN URLS: In some systems, due to bad networking or crackwhore firewalls, people can't use 127.0.0.1 or localhost to access their AmphetaDesk. We should examine ways of detecting the user's IP address on the box (if they have one), and give them the option of using that IP instead (we've instructed people how to hard code their IP address in the past). - PLUGIN APIs. Already planned, and there's a quickie email here on it: http://sourceforge.net/mailarchive/forum.php?thread_id=827854&forum_id=9314 (leonard@cfoknows.com, wkearney99@hotmail.com) - PROXY AUTODISCOVERY: IE registry settings? .pac files? (Autodiscovery of .pac files is documented somewhere. The idea being that a client can make a DNS request for a particularly named file from a specially named host. Searching on .pac should be helpful - Kearney). - RSS PARSING: We really should revamp our RSS parsing system. The current plan is to create AmphetaDesk::Parser which would discover new parsers at AmphetaDesk::Parser::RSSv090.pm, ::RSSv200.pm, and so on. This would allow people to drag and drop new parsers, and AmphetaDesk would understand them at startup. In addition to this, each parser will also return a list of errors concerning why the feed is incorrect (even though we'll be ultraliberal in what we accept, we'll now give the user a chance to report the error). Autodiscovery of parsers should be possible by browsing through %INC for AmphetaDesk::Parser's location, and then getting a readdir of parsers sitting in AmphetaDesk/Parser. When we implement this, our OPML.pm should take over the processing done by AmphetaDesk::ChannelsList::load_channels_by_letter. - RSS PARSING: Namespace support. We'll have to write a "get_prefix_from_url" namespace routine which will take a passed URL (like the NS URL for mod_admin or any of the other RSS modules), and then return the actual prefix used. This will allow our templates to easily print/display that information. - RSS PARSING: Support RDFS descriptions for modules? We'd have to keep a local store of them shipped with AmphetaDesk, but we'll always let those be overridden with data we find online. - SECURITY: There needs to be access lists for the internal webserver - to either allow X, Y, Z ip/dns. We've got the "localhost or not" code, but need some more flexibility. - STDOUT: Check over all the notes and errors and responses. Make sure they're doing what they should be and are well written. Probably a good bet to extract them all and compare them side by side. I'm being really anal about this since it's vitally important to communicate. - TEMPLATES: Look into BROKEN under Text::Template which we can use to display custom errors when there's a template error. This makes a much better output than the blanket Perl error currently. - TEMPLATES: We need to add a front end to downloading and using templates. These templates need to come with info files, so that people can make custom settings, version numbers, and who to contact for more information. - TEMPLATES: Possible ideas for templates: a left-handed bar and click as per Mozilla or IE sidebars. Clicking on the left would open in the main window on the right. A collapsible DHTML interface like shown here: http://www.webreference.com/dhtml/column12/. (jw@abprosys.com) One guy (glutnix@yahoo.com.au) also had us load into IE's search bar: >javascript:Q='';if(top.frames.length==0)Q=document.selection.createRange >().text;void(_search=open('http://127.0.0.1:8888/index.html','_search')) Also, look into deus_ex's outliner version and implement. I've got a Lynx template from Ransom Smith - include that one. See if we can rip off a three-pane NNWL with frames. And finally: http://www.kottke.org/plus/misc/images/netnewswire.gif - TEMPLATES: Figure out why whitespace is needed after our [$ and $] delimiters for Text::Template. This worked fine on simple test cases, but something in our setup causes a necessity for something to be on the delimiter line, even if just a comment. - USAGE DATA: Perhaps allow anonymous upload of AmphetaDesk data that would find out which feeds are most popularly subscribed too, and yadda yadda. This would kinda be a DayPop sorta thing, but more on feeds themselves as opposed to actual items. This lends great credence to an Amazon like recommendation system. (iain@******.net) - USAGE LEARNING / KEYWORDS: A number of people want AmphetaDesk to get smarter. They want to say "These are things I'm interested in", and have AmphetaDesk list those items at the top of the screen, or in a feed all by itself (for website export). The keyword idea would be simple to use - perhaps we could add a button next to each item like "add to keywords". AmphetaDesk would then parse that item, adding uncommon words to the user's keyword list. In some respects, this could be sent to a central server, and AmphetaDesk could recommend new channels that match their interests (although, at this point, this is kinda satanic. Perhaps we could include the relevant data into the massive channel list ourselves?). (sasw@Swernofsky.com) - VERSION CHECKING: Should be an XML file, not plain text. This will all be handled once we jump into Compress::Zlib and start looking into streaming updates. Streaming updates are a long long way off, but at least moving to XML will allow us greater flexibility. We should also move data/internal/version.txt to just plain old data/version.txt, getting rid of the internal/ directory entirely. ------------------------------------------------------------------------------ KNOWN RARE/MINOR BUGS (nothing major, but should be fixed). ------------------------------------------------------------------------------ - MAC AND BROWSER CHOOSING: Mac users can't choose a browser besides the default. Alan has sent a MacEvent solution, we should try and integrate that. [This has never been brought up so is low priority.] - MAC OS X AND MISSING IMAGE: "Just one question: there's a large empty area at the top of the AmphetaDesk OS X window (above the status line that you can 'dink' open to see the log)." (withers@optushome.com.au; screenshot available. ask morbus). Using 10.1.5 and the April Dev Tools. Confirmed using Jaguar as well.