Removing Duplicates from Return Separated List

Here are few examples of removing duplicates from a return separated file. The return character is the Unix \n character. If your file is another format you can either change the character in the examples or change the character in your file with the following code.

dos2unix:  tr -d \\r < dosfile > unixfile
mac2unix:  tr \\r \\n < macfile > unixfile

Do Shell Script from AppleScript Example

do shell script "tr '\\r' '\\n' <" & posixInput & " | sort -fu > " & posixOutput

Ruby Example Usage:
uniq_from_file('/path/to/input.txt', '/path/to/output.txt')

#!/usr/bin/env ruby -w

def uniq_from_file(input, output)
  txt = IO.read(input)
  clean = txt.split(/\n/).uniq
  File.open(output, 'w') { |f| f.puts clean }
end

Objective-C Foundation Tool Usage:
removeDups originalFilePath destinationFilePath

#import <Foundation/Foundation.h>

int main (int argc, const char * argv[]) {
    NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
	
	NSArray *args = [[NSProcessInfo processInfo] arguments];
	
	if ( argc != 3 ) {
		printf("Usage: \n\tremoveDups originalFilePath destinationFilePath");
		return 1;
	}
	
	NSString		*path = [args objectAtIndex:1];
	NSString		*destPath = [args objectAtIndex:2];
	NSString		*fileAsString = [NSString stringWithContentsOfFile:path];
	
	NSArray *lines = [fileAsString componentsSeparatedByString:@"\n"];
	NSSet	*uniques = [NSSet setWithArray:lines];

	NSArray *sortedArray = [[uniques allObjects] sortedArrayUsingSelector:@selector(caseInsensitiveCompare:)];
	NSString *cleanListString = [sortedArray componentsJoinedByString:@"\n"];
	
	if ( [cleanListString writeToFile:destPath atomically:YES encoding:NSUTF8StringEncoding error:nil] == NO ) {
		printf("Error writing clean list to file");
		return 2;
	}
	
	[pool drain];
	return 0;
}