If I have a string “1 2 3 2 1” – or an array [1,2,3,2,1] – how can I select the unique values, i.e.
"1 2 3 2 1" produces "1 2 3"
or
[1,2,3,2,1] produces [1,2,3]
Similar to uniq but uniq seems to work on whole lines, not patterns within a line…
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
If you are using zsh:
$ array=(1 2 3 2 1)
$ echo ${(u)array[@]}
1 2 3
or (if KSH_ARRAYS option is not set) even
$ echo ${(u)array}
1 2 3
Method 2
With GNU awk (this also retains original order)
printf '%sn' "1 2 3 2 1" | awk -v RS='[[:space:]]+' '!a[$0]++{printf "%s%s", $0, RT}'
1 2 3
To read into a bash array
read -ra arr<<<$(printf '%sn' "1 2 3 2 1" |
awk -v RS='[[:space:]]+' '!a[$0]++{printf "%s%s", $0, RT}')
printf "%sn" "${arr[@]}"
1
2
3
Method 3
For an array with arbitrary values, it’s quite tricky with bash as it doesn’t have a builtin operator for that.
bash however happens not to support storing NUL characters in its variables, so you can make use of that to pass that to other commands:
The equivalent of zsh‘s:
new_array=("${(@u}array}")
on a recent GNU system, could be:
eval "new_array=($(
printf "%s" "${array[@]}" |
LC_ALL=C sort -zu |
xargs -r0 bash -c 'printf "%qn" "<a href="https://getridbug.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="163256">[email protected]</a>"' sh
))"
Alternatively, with recent versions of bash, and assuming none of the array elements are empty, you could use associative arrays:
unset hash
typeset -A hash
for i in "${array[@]}"; do
hash[$i]=
done
new_array=("${!hash[@]}")
With bash 4.4 and newer and with GNU sort:
readarray -td '' new_array < <(
printf '%s' "${array[@]}" | LC_ALL=C sort -zu)
The order of the elements would not be the same in those different solutions.
With tcsh:
set -f new_array = ($array:q)
Would retain the first element (a b a => a b) like zsh‘s (u) expansion flag.
set -l new_array = ($array:q)
Would retain the last (a b a => b a). Those however remove empty elements from the array.
Method 4
This solution worked for me.
ids=(1 2 3 2 1)
echo "${ids[@]}" | tr ' ' 'n' | sort -u | tr 'n' ' '
The above produces 1 2 3 as the output.
Shorter version as suggested by Costas could be,
printf "%sn" "${ids[@]}" | sort -u | tr 'n' ' '
To store the end results to an array, you could do something like,
IFS=$' '
arr=($(printf "%sn" "${ids[@]}" | sort -u | tr 'n' ' '))
unset IFS
Now, when I do an echo on arr, this is the output I get.
echo "${arr[@]}"
1 2 3
References
https://stackoverflow.com/a/13648438/1742825
https://stackoverflow.com/a/9449633/1742825
Method 5
To do it entirely in the shell and put the result in an array,
declare -A seen
for word in one two three two one
do
if [ ! "${seen[$word]}" ]
then
result+=("$word")
seen[$word]=1
fi
done
echo "${result[@]}"
In words: if we haven’t seen a given word yet,
add it to the result array and flag it as having been seen.
Once a word has been seen, ignore subsequent appearances of it.
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0