1

We have a list(s) of URLs, the task is to test all the URLs:

  • they should return proper response code which is configured somehow it the list
  • some should return content having a specified keyword in it
  • some should be checked considering redirects, some not.
  • they should be fast enough to fit in a defined timeframe
  • etc

The question: is there any ready-to-use tool that allows just to pass the properly formatted list of URLs as input and get the pass/fail answer?

The first tool I can think about to do the job is a wrapper around curl, however, we would need to write a script to parse list/parameters (expected response code, optional keyword, whether to follow redirects, etc). I can write it on my own but I'm new in testing so there definitely is something I haven't thought of about yet but which might be in that tool already.

No GUI/rendering analyze required, the process will be run in CI/CD pipeline i.e. in fully detached/automated mode without access to any graphical interface. Linux.

4
  • 3
    What about Postman? – Mate Mrše Sep 29 '20 at 8:30
  • 1
    curl, postman with newman, python with request library, ... You choose. In my experience, curl is not very comfortable to use, but it could be done. – pavelsaman Sep 29 '20 at 9:32
  • @MateMrše Postman itself afaik cannot run headless, this is why I didn't consider it. So the answer would be Postman+newman, if my understanding is correct. If you post it, I'll accept it. – Putnik Sep 29 '20 at 12:46
  • 1
    @putnik who told you cannot run postman without UI ,Newman is a command line utility to run postman scripts . You dont need postman installed in the target system – PDHide Sep 29 '20 at 13:05
3

Postman has a command line Collection Runer called Newman:

By default, Newman exits with a status code of 0 if everything runs as expected with no exceptions. You can configure your continuous integration tools to respond to Newman's exit codes and correspondingly pass or fail a build.

You can easily integrate it with

1

If you go the bash route The following may help.

It uses a function called do_test that let you pass a function and test the results.

#!/usr/bin/env bash
. colors_for_text.sh 2> /dev/null

do_test () {
  if [[ $# -eq 0 ]]; then
    printf "$fail_color Error - do_test: No parameters provided - Usage is do_test [function_name] \$LINENO [expected result] [optional params]\n"
    exit 1
  elif [[ $# -eq 1 ]]; then
    printf "$fail_color Error - do_test: Two parameters missing - \$LINENO and expected result (other params as needed)\n"
    exit 1
  fi
  declare -ig test_runs test_passes test_fails
  local -r function="$1";shift
  local -r line_number="$1";shift
  local failfast="False"
  local debug="False"
  local verbose="False"
  local test_description=
  local test_matcher=
  local expected=
  local pass_on_params=
  local command_not_found=False
  type "setup" &> /dev/null && setup
  # TODO Replace with for loop (initial attempts failed)
  [[ "$1" == "failfast" ]] && failfast="True" && shift
  [[ "$1" == "debug" ]] && debug="True" && shift
  [[ "$1" == "verbose" ]] && verbose="True" && shift
  [[ "$1" == "failfast" ]] && failfast="True" && shift
  [[ "$1" == "debug" ]] && debug="True" && shift
  [[ "$1" == "failfast" ]] && failfast="True" && shift
  if [[ "$1" =~ ^description= ]]; then
    [[ `echo $1 | grep =` ]] && test_description=`echo $1 | sed 's/.*=//'`' ' &&
    shift
  fi
  if [[ "$1" =~ ^matcher= ]]; then
    [[ `echo "$1" | grep =` ]] && test_matcher="`echo "$1" | sed 's/matcher=//'`" &&
    shift
  fi
  if [[ $# -eq 0 ]]; then
    printf "$fail_color Error - Function: $function, Line: $line_number \n
do_test: Third parameter missing - expected result\n"
    exit 1
  fi
  expected="$1";shift
  pass_on_params="$@"
  $function ${pass_on_params[@]} 2> $TMPDIR"/output$$-"$test_runs".txt"
  cat $TMPDIR"/output$$-"$test_runs".txt" | grep -i "command not found" &&
    command_not_found=True
  debug_printf "(debug)"
  if [[ $command_not_found == "True" ]]; then
    record_script_failure "$function"
  elif [[ "$test_matcher" == "==" ]]; then
    [[ "$result" == $expected ]] &&
    record_test_success "$function" ||
    record_test_failure "$function";
  elif [[ "$expected" =~ ^-?[0-9]+$ ]]; then
    [[ "$result" -eq $expected ]] &&
    record_test_success "$function" ||
    record_test_failure "$function";
  else
    [[ "$result" == $expected ]] &&
    record_test_success "$function" ||
    record_test_failure "$function";
  fi
  test_runs+=1
}
record_test_success () {
  [[ $# -eq 0 ]] && no_param_quit $FUNCNAME
  local -r function="$1"
  printf "$pass_color"."$color_end"
  [[ "$verbose" == "True" ]] && printf "$function"
  test_passes+=1
}
record_test_failure () {
  [[ $# -eq 0 ]] && no_param_quit $FUNCNAME
  local -r function="$1"
  test_fails+=1
  printf "$fail_color"F"$color_end"
  function_exists? "$function" &&
    error_messages=$error_messages"$fail_color""$test_description""Line: $line_number - \
'$function ${pass_on_params[@]}' failed:\n\
Expected: $expected\n\
Received: $result $color_end\n\n" ||
    error_messages=$error_messages"$fail_color""Line: $line_number: \
Function '$function' is undefined\n"
  [[ "$failfast" == "True" ]] && failfast "Test" $error_messages
}
record_script_failure () {
  [[ $# -eq 0 ]] && no_param_quit $FUNCNAME
  local -r function="$1"
  test_fails+=1
  err=`cat $TMPDIR"/output$$-"$test_runs".txt"`
  printf $err
  error_messages=$error_messages"$fail_color""Line: $line_number: \
Error within function '$function', getting \"command not found\" in $err \n"
  [[ "$failfast" == "True" ]] && failfast "Script" $error_messages
}
failfast () {
  printf "\n$1 error & failfast set to exit immediately.\n$2\n" && exit 1
}

no_param_quit () {
  printf "\nError - 1 parameter required for: $1 - exiting\n" && exit 1
}
function_exists?() {
    declare -f -F $1 > /dev/null
    return $?
}
debug_echo () {
  [[ "$debug" == "True" ]] && echo "$1"
}
debug_printf () {
  [[ "$debug" == "True" ]] && printf "$1"
}
13:11:33 durrantm Castle2012 /home/durrantm/Dropbox/_/sh__sed__awk/euler 
$

For usage see the bottom of https://durrantm.wordpress.com/2015/04/02/tdd-for-sh/

1

Just to add to the above answers,

in postman if you click the collection and click view in web you can see the equavalent curl commands:

enter image description here

Then you can see curl comment in the api documentation :

enter image description here

Postman also supports retrieving code snippets in different programming languages:

Click the specific request and then click code:

enter image description here

Then you can see code as below , you can select which programming language you wnat the code generated for from the list :

enter image description here

1

You can also use Insomnia for manual testing.

I have previously used Soap UI as well and automated this more.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.