Hello there, replying inline

> Any other ideas?

Hyper Cache

> >> Hello Fajar,
> >>
> >> I would like to say hello and get in contact with you.

Hello there, nice to get in contact with you too. Greeting from Indonesia.

> >> Michael from IPFire.org told me that you may have some requirements or
> >> ideas for an improved update accelerator

Firstly, this idea is not part of update accelerator thingie.

> >> cause we plan to extend the
> >> current version (at this point it look like a complete rewrite o_O)

complete rewrite, maybe :)

My idea is basically out from squid 2.7 abilities to cache dynamic contents using built-in storeurl feature.
http://www.squid-cache.org/Doc/config/storeurl_rewrite_program/

Wiki for example how to use storeurl
http://wiki.squid-cache.org/Features/StoreUrlRewrite

We already know that squid 2.7 is already obsolete - but this feature was extremely useful for slow internet users (just like me in Indonesia, where bandwidth is expensive). Built-in storeurl feature inside squid 2.7 has ability to cache or manipulating caching for dynamically hosted contents (dynamic contents). Example for Facebook's CDN :

http://profile.ak.fbcdn.net/hprofile-ak-prn1/s160x160/572710_100002038935636_1362357155_n.jpg
http://profile.ak.fbcdn.net/hprofile-ak-prn2/s160x160/572710_100002038935636_1362357155_n.jpg
http://profile.ak.fbcdn.net/hprofile-ak-prn3/s160x160/572710_100002038935636_1362357155_n.jpg


Hosted image/content is identically same and statically hosted in different Facebook's CDN servers - Michael's pic :)


If squid already cache one of this picture then all same pictures from hprofile-ak-prn1, hprofile-ak-prn2, hprofile-ak-prn3 ..... hprofile-ak-prnX will result cache hit - squid not necessary to fetch same content from different cdn urls, since its already in cache and request got rewritten by storeurl. All contents from Facebook such as javascript, css, image, even sound and videos will have very high chance to get hits from squid.



This method works on almost all web that serving dynamic contents for its visitors : Youtube videos (all resolutions) , blogger.com contents, online games patch files, google maps, ads, imeem, etc. This is something that you cannot done with squid 3.x.


Configuration and storeurl example to cache Youtube and some CDN (works for me until now -  please read the comments too):
http://tumbleweed.org.za/2009/02/18/fun-squid-and-cdns


This one written in perl, fbcdn seems not work here.

http://code.google.com/p/tempat-sampah/source/browse/storeurl.pl?r=17


Development for same feature for squid 3 is on the way
http://wiki.squid-cache.org/Features/StoreID


Another approach to make it work on squid 3 is using ICAP - I'm not familiar with this one since I never used it. You can see some reference about ICAP to cache dynamic contents here (for me it seems difficult to do it) : http://www.squid-cache.org/mail-archive/squid-users/201206/0074.html


All objects will be cached or purged following refresh_pattern directives.



> >> So how is your current situation?

> >> It would be great if you can give some details of your current system,
> >> your users, for what you use the update accelerator, the kind of files
> >> you currently store and the space it currently used.

I used IPFire to serve office and wireless internet clients (3 mbps ADSL shared with 20 - 60 clients, you can expects torrent, and other bandwidth hunger users. duh!). Its just works like a charms. Update accelerator also comes handy - it help to boosts some antivirus and OS'es. Adding extra file extension .vpx - Avast 7.x updates - its use by most of my wireless users. I used 1 TB harddisk for proxy cache and update accelerator.

> >> What could be improved or changed in an upcoming version?

Dynamic contents caching ability inside IPFire.

> >> What is disturbing you at moment?

Bandwidth limitation is our biggest problems - not only for me, but also most internet users in Indonesia.

Sorry for the late response, and thanks for your kind attention. Terima kasih (thank you - in Bahasa Indonesia).

- Fajar

http://www.ipfire.web.id/