$ awk '{print | "sort ..."} $ awk '{print ...}' | sort $ awk '!a[$0]++' $ awk 'line !=$0{line=$0}'