I am starting the development of a web platform that provides automated tasks to users, and I have been redirected to this forum after asking the same question in stackoverflow and not being accepted as it is opion based, which is exactly what I am looking for.

I have no technology requirements, and I am tackling as to what should be the best choice regarding the technology and platform I will be implementing this in, as ultimately I am concern about the integration of the web application itself with the features or applications that allow me to do tasks on specific sites on behalf of users.

I can do this with PHP + Laravel and use some external package such as Goutte for handling the scraping and interaction, I can rely on Selenium and interact with it through php or by calling selenium directly as long as I have the ability to execute applications using exec(), system() or something else.

I can also go with Python + Django and use this as the scraper, or once again rely on some external application installed in the system such as Selenium, or go with something else.

My main concern is limitation due to the chosen technology, as I will have to process output of the pages, and I don't really like system or exec in php and then trying to parse their output, and I would appreciate it if someone with prior experience would shed some light into caveats of any of these, or alternative suggestions.

I understand that this might be a personal perspective, but I welcome those, as it would mean people have bumped into issues, or tried different things. If this is not the correct stackexchange forum for this, please let me know so I can ask this somewhere more appropriate.

  • I'm voting to close this question as off-topic because it is unrelated to software testing or software quality.
    – user246
    Commented Nov 16, 2017 at 12:22
  • @user246 thank you. I was redirected here by another user, so I would ask you the same thing I asked him. Which forum in stackexchange would be appropriate for such a question? Commented Nov 16, 2017 at 12:25
  • 2
    None of the stackexchange sites are good places to ask for opinions. You would do better to seek out a testing discussion board rather than a question-and-answer site.
    – Kate Paulk
    Commented Nov 16, 2017 at 12:40
  • Maybe Software Recommendations can help, but you should be more specific.
    – beatngu13
    Commented Nov 17, 2017 at 20:35