The input is a csv where the first column is the URL you want to test. The curl line spits out the response code. -f means curl will return an exit code if the fetch fails.


# test URLs
for x in $(cat "$file" | grep -v URL | cut -d, -f1) ; do code=$(curl -s -w %{response_code} -f "$x" -o /dev/null) && echo "URL OK $code: $x";  done

# convert URLs to nginx redirects
sed -e 's#^http://[^/]*\(/[^,]*\),\(.*\)$#rewrite ^\1$ \2 permanent;#' < "$file" > /tmp/output_redirects


Good times.