You don't have to retrieve all content of a URL to determine the page is valid or not, just get the status code from URL's header. Therefore, you need a function to handle it. In this post, I demonstrate a function in PHP for an example of retrieving the HTTP status code of URL, you may use it for your applications.
function validateURL($url) {
$exists = false;
$headers = @get_headers($url);
if ($headers && substr($headers[0], 9, 3) < 400) {
$exists = true;
}
return $exists;
}
As you can see, the codes greater than 400 will return false, this is because they are defined as unhealthy status. You may check the following status codes that are greater than 400.
Client Errors: HTTP Code 4xx
4xx are all client errors:
400 Bad Request
401 Unauthorized
402 Payment Required
403 Forbidden
404 Not Found
405 Method Not Allowed
406 Not Acceptable
407 Proxy Authentication Required
408 Request Timeout
409 Conflict
410 Gone
411 Length Required
412 Precondition Failed
413 Request Entity Too Large
414 Request-URI Too Long
415 Unsupported Media Type
416 Requested Range Not Satisfiable
417 Expectation Failed
401 Unauthorized
402 Payment Required
403 Forbidden
404 Not Found
405 Method Not Allowed
406 Not Acceptable
407 Proxy Authentication Required
408 Request Timeout
409 Conflict
410 Gone
411 Length Required
412 Precondition Failed
413 Request Entity Too Large
414 Request-URI Too Long
415 Unsupported Media Type
416 Requested Range Not Satisfiable
417 Expectation Failed
Server Errors: HTTP Code 5xx
5xx are all server errors:
500 Internal Server Error
501 Not Implemented
502 Bad Gateway
503 Service Unavailable
504 Gateway Timeout
505 HTTP Version Not Supported
501 Not Implemented
502 Bad Gateway
503 Service Unavailable
504 Gateway Timeout
505 HTTP Version Not Supported
In practice, I like to return the status code instead of its existence (true or false) because it's more flexible for further logics.