| Author |
Is there a web page over 500Kb? |
sadeghi85
Joined: Oct 13, 2007
Posts: 341
PM |
Is there a web page over 500Kb? (html only, excluding images, flash, etc etc) |
|
|
jcwhite_uk
Joined: Feb 18, 2004
Posts: > 500
From: Dorset, UK Phone:Xperia Z1
PM, WWW
|
I doubt it. 500kb is a lot of HTML code
Checkout my photos at My Website"Duct tape is like the force. It has a light side, a dark side, and it holds the world together." |
se_p800
Joined: Jan 07, 2004
Posts: 273
From: England
PM, WWW
|
Why do you want to know?
Like playing Call of Duty?
Then sign up to www.forumwarfare.com! |
sadeghi85
Joined: Oct 13, 2007
Posts: 341
PM |
I'm currently developing an app with PHP and i want to know what's the maximum size of regular web pages, so i can set a limit to not downloading web pages but interested files.("content type" won't work cos for example metacafe server will send "text/plain" for "flv" files that obviously they aren't plain text) |
Johnex
Joined: Nov 26, 2002
Posts: > 500
From: Stockholm/Sweden
PM, WWW
|
Well, i usually serve all my files through php so i can limit the speed. Works for very large files, like 200mb+. The problem is that php has a timeout limit of 30seconds, depending on your server configuration.
[ This Message was edited by: Johnex on 2008-01-13 13:59 ] |
sadeghi85
Joined: Oct 13, 2007
Posts: 341
PM |
It's not a good decision IMO. client can't use "resuming" feature cos web server can't send "Content-Range" header, unless you implement this feature directly in php script that is hard( and why we should do that when the web server already supports it). consider a client on a slow connection(for example firefox trying to download an image) if it can't completely download that image, it will send another request with "Range" header to download remainder bytes. if you pass your files with php, you force client to download that file from beginning, overwhelming server.
regarding timeout limit you can use: @set_time_limit(0);
if "safe mode" is off. if you can access php.ini, set "max_execution_time" appropriately.
my point wasn't to limit speed.
EDIT:
Additionally this approach will prevent "caching" mechanism. client will send "If-Modified-Since" and server can't create "304 Not Modified", although the file is untouched, and again overwhelming server with unnecessary requests.(and my browser should download that pic every time i visit this page, so remove that pic please :) )
[ This Message was edited by: sadeghi85 on 2008-01-12 23:01 ] |
Johnex
Joined: Nov 26, 2002
Posts: > 500
From: Stockholm/Sweden
PM, WWW
|
Dude, i know all this, i have worked with php for 6 years. I gave that as an example, since you want a site to be sent with php. |
sadeghi85
Joined: Oct 13, 2007
Posts: 341
PM |
Well, my second post was confusing. let me explain my situation again. i want to use php as a client, since C++ is too complicated for me to use. my app is like a downloader. it searches a web page for links and download certain files. i want it to download all files except those that are plain text in nature. since "Content-Type" won't work, i need a size limit so if a file is bigger than that limit it can be considered as a binary file. till now a limit of 500Kb worked for me. biggest html file i saw was about 350Kb somewhere on mp3.com.
and i'm curious why you want to limit speed? |
ÈL ® ö B ì Ñ
Joined: Feb 03, 2005
Posts: 281
PM |
Because you don't want 5 people downloading large files and maxing out the server connection and having the site run slower for everyone else just doing general browsing?
Just as an example.
|
Cycovision
Joined: Nov 30, 2003
Posts: > 500
From: England
PM, WWW
|
You could always write a little PHP string handing function to find the dot(s) and determine the filename's extension that way?
|
sadeghi85
Joined: Oct 13, 2007
Posts: 341
PM |
That way won't work. consider a link like this: http://example.com/?fileid=35465
it can be a file and there is no dot on it. so i should check "response header", and "content-disposition" isn't always available.
On 2008-01-13 18:35:41, ÈL ® ö B ì Ñ wrote:
Because you don't want 5 people downloading large files and maxing out the server connection and having the site run slower for everyone else just doing general browsing?
Just as an example.
using two host, one for large file storage and the other for general use is a better approach IMO. i stated disadvantages of passing file with php before. |
Johnex
Joined: Nov 26, 2002
Posts: > 500
From: Stockholm/Sweden
PM, WWW
|
ÈL ® ö B ì Ñ is correct on my usage. Also, making it so members have faster download speeds is something that rapidshare uses, and works quite nicely, incentive to register.
I would go with what cyco suggested, but dont see how php will download multiple files for you, well, to the webserver its possible. I made a php based proxy a while back, and it opens a socket connection to a page, downloads the page, and edits all the links to go through the proxy. All files that are linked also go through the proxy without issues, gives the correct content type. This could be modified to use the socket to download to the actual server, but it wont be as effective as having a native c++ or c program.
[ This Message was edited by: Johnex on 2008-01-13 19:21 ] |
sadeghi85
Joined: Oct 13, 2007
Posts: 341
PM |
Rapidshare encourages users to get premium account by 3 strategies:
1. by passing the file through php(if they are using php):
*user wants to download a file. if he faces a connection drop he should download the file from beginning.
**he can't use a downloader(e.g firefox->flashgot->flashget) because downloader can't use the socket opened by browser and should open another socket and php script will recognize it as a separate request.
2. by blocking ip for e.g one hour.
3. by using CAPTCHA image to prevent automation.
these restrictions are annoying and force users to register. a regular download using a downloader like flashget is even faster. so passing file through php is to force users to register not for having faster download speeds for all.
actually i wrote the app for downloading from Rapidshare! (and other hosts that don't support resuming download).
till now my approach has been successful. i haven't any problem with the app. i just asked "Is there a web page over 500Kb?"
thanks for replies. |
Johnex
Joined: Nov 26, 2002
Posts: > 500
From: Stockholm/Sweden
PM, WWW
|
http://pizzaseo.com/google-cache-maximum-file-size
Short and concise, yes there are. |
sadeghi85
Joined: Oct 13, 2007
Posts: 341
PM |
Thanks.
since Google is unable to go over 1MB cache limit, i'll set the limit to 1MB.
many many thanks, you solved my problem  |
|
|
Posted: 2008-01-12 03:40
