How to write a java program to find unique patterns for API URLs?

Note: I have a file which contains 10000 API URLs with duplicates



Unique patterns means removing duplicates like below:



For this:

  • 1
    I'm really not sure what are you asking here, what are unique patterns ? why Java ?
    – Rsf
    Sep 14, 2017 at 15:02
  • What is your level of Java skills? Do you know how to do trivial task like write text to a file? If it is not obvious, you might need to work on your java, following some tutorials. Sep 14, 2017 at 15:04
  • Unique patterns means i need to remove duplicates(i have a file which contains 10000 urls ).
    – lak
    Sep 14, 2017 at 15:46
  • Des it have to be java? I can give you pseudocode easily enough. Does performance matter? If you have access to a unix system, look at "sort | uniq" Sep 14, 2017 at 19:09
  • Still not quite sure what you mean when say a "duplicate". What is the rule one shoould use to decide if there is a dup record they're looking at.
    – Alexey R.
    Sep 15, 2017 at 15:50

1 Answer 1


This is the answer working fine for me

public class LineMatcher {
    static Set<String> linkedHashSet = new LinkedHashSet<>();

    public void findBadLines(String aFileName) {
        Path path = Paths.get(aFileName);
                        BufferedReader reader = Files.newBufferedReader(path, ENCODING);
                        LineNumberReader lineReader = new LineNumberReader(reader);
                ) {
            String line = null;
            while ((line = lineReader.readLine()) != null) {
                line = line.concat("&");
                Pattern pattern = Pattern.compile("=[^&]*&");
                Matcher m = pattern.matcher(line);
                while (m.find()) {
                    String str = m.group();
                    String cc = str.substring(1, str.length() - 1);
                    if (cc.isEmpty()) break;
                    line = line.replace(cc, "");
                    line = line.replace("=&", "+");
                    m = pattern.matcher(line);
        } catch (IOException ex) {

 public static void main(String... arguments) {

LineMatcher lineMatcher = new LineMatcher();
lineMatcher.findBadLines("C:\\Users\\Desktop\\New folder\\api.csv"); 

for (String i : linkedHashSet) 


Out Put(removing duplicates):




Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.