Aimy Sitemap 3.7.0 is out and available to the Joomla! community!
So, what's new in this release?
Short answer: we have rewritten the crawler to support even large websites and improved its user interface as well.
All users of Aimy Sitemap are encouraged to update to the new version.
Long answer: We have been using Aimy Sitemap for customer websites for a long time until we made the step to finally release it and give it to the community for free. Most of our customers run small to medium websites - so we initially designed Aimy Sitemap for them and with their needs in mind.
That actually turned out being inadequate for large websites, where Aimy Sitemap sometimes did fail to crawl as expected due to resource exhaustion issues. These were caused by the requirement of a single, long running PHP process that did all the crawling.
The new release does not come along with this requirement anymore.
Within the last ten days Aimy Sitemap has grown to run on a couple of hundred Joomla! installations already, so changing shoes has been necessary to not get out of step!
Crawler
Aimy Sitemap's new crawler analyzes your website one URL after another, using one PHP process for each URL. Resource exhaustion issues are therefore more than just unlikely - if not impossible at all.
Besides that, the whole interaction between crawler and backend user interface has been rewritten nearly from scratch. The new version not only provides a detailed crawl report, but also just-in-time information on the crawl in progress:
The crawling report has also been reworked. The report is now splitted in two columns, showing all all results grouped by the action that has been performed on your index on the left side, listing all affected URLs as a list for quick review. Any crawling error, that might have occurred is shown as an alert box in the right column. This information allows you to easily spot any dead links, broken redirects and stuff like that.
Have a look at an example screenshot of a crawling report:
Various Minor Improvements
With this release, some minor improvements are waiting for you as well:
- a new default exclude pattern has been added:
*/assets/* - URLs are now sorted ascending by name in "Manage URLs", previously they had been sorted descending by change-time
- page title extraction now even supports overridden
JPAGETITLEconstants - the default crawling delay has been changed to zero seconds, as the new crawling algorithm does use less resources at once
Compatibility Notes
As we had to rework the crawler's design to support larger Joomla! installations as well, we had to take a somewhat fresh start in this release.
Based on dozens of interactions with the Joomla! community we therefore decided to drop some of the previous design decisions and changed some internals to better fit the reported requirements.
These changes won't affect most users, but may impose some extra work for a few users:
Non-Search Engine Friendly Websites
Aimy Sitemap's crawler now internally uses Joomla!'s JURI class to handle
all crawled URLs. Previously, a class of ours had been used.
JURI automatically encodes query strings in raw format, while Aimy
Sitemap's class used entity-encoded ones.
As a result, after an upgrade on a non-SEF website, the URLs' encoding will change. Therefore all previously known non-SEF URLs will be deleted once and replaced with their equivalent raw version (that is, using "&" instead of "&" internally). This leads to the requirement to reconfigure any previously customized URLs.
Users of SEF-enabled websites are not affected!
Reporting of Malformed Links
With the release of Aimy Sitemap v3.6.2 we introduced the feature to report
malformed links to the user in the "Manage URLs" view. With the switch to
JURI this feature had to be dropped, because the required information is
lost due to JURI's automatical conversion feature.
So, why did we do that? We try to stay as close as possible to Joomla!'s
code base for various reasons. JURI automatically encodes all URLs to
a uniform representation which is actually a large benefit. Besides that,
a Joomla! release's code has already been tested by both developers and the
community and is considered stable. So, by using as much of the powerful
tools Joomla! provides we save time that we use to improve our extensions'
functionality intead of reinventing the wheel.
Aimy Sitemap primarily strives to provide you an easy all-in-one sitemap solution, and not a website validator.
What's on the Horizon?
After Aimy Sitemap has proven to work on large websites as well, we will consider starting to work on some new features:
- Aimy Canonical support for automatical Canonical URL handling
- Document language extraction and filtering
- Editable titles for each URL (to be shown in HTML sitemaps)
- Automatical elimination of duplicate titles (for HTML sitemaps)
- Selectively enable/disable URLs for HTML sitemaps as well
- Improved change-time handling
- Hierarchical HTML sitemap support
- Live-Sitemap support (generating non-disk based XML sitemaps on request)
- PostgreSQL support
- Custom CSS page classes for HTML sitemaps
And as usual: we're open to feedback and appreciate your feature requests!
Thanks to all users who reported issues and helped to make Aimy Sitemap even better!
Read on about what changed in Aimy Sitemap v3.6.2.