How to write a java program to find unique patterns for API URLs?

Note: I have a file which contains 10000 API URLs with duplicates



Unique patterns means removing duplicates like below:



For this:

  • 1
    I'm really not sure what are you asking here, what are unique patterns ? why Java ? – Rsf Sep 14 '17 at 15:02
  • What is your level of Java skills? Do you know how to do trivial task like write text to a file? If it is not obvious, you might need to work on your java, following some tutorials. – Peter M. - stands for Monica Sep 14 '17 at 15:04
  • Unique patterns means i need to remove duplicates(i have a file which contains 10000 urls ). – lak Sep 14 '17 at 15:46
  • Des it have to be java? I can give you pseudocode easily enough. Does performance matter? If you have access to a unix system, look at "sort | uniq" – Kevin McKenzie Sep 14 '17 at 19:09
  • Still not quite sure what you mean when say a "duplicate". What is the rule one shoould use to decide if there is a dup record they're looking at. – Alexey R. Sep 15 '17 at 15:50

This is the answer working fine for me

public class LineMatcher {
    static Set<String> linkedHashSet = new LinkedHashSet<>();

    public void findBadLines(String aFileName) {
        Path path = Paths.get(aFileName);
                        BufferedReader reader = Files.newBufferedReader(path, ENCODING);
                        LineNumberReader lineReader = new LineNumberReader(reader);
                ) {
            String line = null;
            while ((line = lineReader.readLine()) != null) {
                line = line.concat("&");
                Pattern pattern = Pattern.compile("=[^&]*&");
                Matcher m = pattern.matcher(line);
                while (m.find()) {
                    String str = m.group();
                    String cc = str.substring(1, str.length() - 1);
                    if (cc.isEmpty()) break;
                    line = line.replace(cc, "");
                    line = line.replace("=&", "+");
                    m = pattern.matcher(line);
        } catch (IOException ex) {

 public static void main(String... arguments) {

LineMatcher lineMatcher = new LineMatcher();
lineMatcher.findBadLines("C:\\Users\\Desktop\\New folder\\api.csv"); 

for (String i : linkedHashSet) 


Out Put(removing duplicates):




Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.