SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Remove items from an array based on another array


Remove items from an array based on another array

Author
Message
schleep
schleep
SSCrazy
SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)

Group: General Forum Members
Points: 2823 Visits: 1406
Hi,

I have 2 Array or ArrayList objects, with data like


$Results = New-Object system.Collections.ArrayList
$Results.Add("IMP_Trn_Project_Agent_6,User1")
$Results.Add("IMP_Trn_Project_Agent_6,User2")
$Results.Add("IMP_Trn_Project_Agent_6,User3")

$Remove = New-Object system.Collections.ArrayList
$Remove.Add("User2")



How can I efficiently remove from the first array all items containing the users in the 2nd array?

Paul



bruce 1565
bruce 1565
Mr or Mrs. 500
Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)

Group: General Forum Members
Points: 543 Visits: 1031
Here's one way, there are probably more.
foreach ($result in $results) {if ($remove -contains $result.split(",")[1]) { $results.Remove($result) }}


schleep
schleep
SSCrazy
SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)

Group: General Forum Members
Points: 2823 Visits: 1406
I tried your code, it gives me an error:

An error occurred while enumerating through a collection: Collection was modified
; enumeration operation may not execute..
At ps1:10 char:8
+ foreach <<<< ($result in $results) {if ($remove -contains $result.split(",")[1
]) { $results.Remove($result) }}
+ CategoryInfo : InvalidOperation: (System.Collecti...numeratorSimp
le:ArrayListEnumeratorSimple) [], RuntimeException
+ FullyQualifiedErrorId : BadEnumeration


Also, I modified the $Results array to add a second instance of User2: it doesn't get removed, while I need it removed as well.


$Results = New-Object system.Collections.ArrayList
$Results.Add("IMP_Trn_Project_Agent_6,User1")
$Results.Add("IMP_Trn_Project_Agent_6,User2")
$Results.Add("IMP_Trn_Project2_Agent_6,User2")
$Results.Add("IMP_Trn_Project_Agent_6,User3")

$Remove = New-Object system.Collections.ArrayList
$Remove.Add("User2")

foreach ($result in $results) {if ($remove -contains $result.split(",")[1]) { $results.Remove($result) }}

$Results





bruce 1565
bruce 1565
Mr or Mrs. 500
Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)

Group: General Forum Members
Points: 543 Visits: 1031
I must have changed the ErrorActionPreference in the console in which I tested my answer. I didn’t get the error last week but do today.
Anyway, give this version a try. I don’t get errors running with ErrorActionPreference set to Continue.
$Results = New-Object system.Collections.ArrayList
$Results.Add("IMP_Trn_Project_Agent_6,User1")
$Results.Add("IMP_Trn_Project_Agent_6,User2")
$Results.Add("IMP_Trn_Project2_Agent_6,User2")
$Results.Add("IMP_Trn_Project_Agent_6,User3")

$Remove = New-Object system.Collections.ArrayList
$Remove.Add("User2")

for ($i=($Results.Count -1); $i -ge 0; $i--) {
if ($results[$i].split(",")[1] -eq $remove) { $results.RemoveRange($i,1) }
}
$Results


schleep
schleep
SSCrazy
SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)

Group: General Forum Members
Points: 2823 Visits: 1406
Thanks again Bruce.

I got it working using a foreach $Result (If not in $Remove, add to $FinalResult) approach.
Works reasonably fast. (13K $Results to check against a list of 500 $Removes)

Cheers!

Paul



bruce 1565
bruce 1565
Mr or Mrs. 500
Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)

Group: General Forum Members
Points: 543 Visits: 1031
OK, that was one of those "other ways" I initially mentioned but I was trying to stay with your original request. Either way I'm glad you got it working.
However, if you find some extra time could you first verify my last reply actually does what you wanted and also compare the time your version and mine take? Or if it's easier to send files with 13K results and 500 removers I'd be glad to do the comparison myself.
Not like it's that important since you've answered your question. I'm just curious about which is more efficient in regard to speed and resources. I'm guessing creating the $FinalResult array is quicker.
schleep
schleep
SSCrazy
SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)SSCrazy (2.8K reputation)

Group: General Forum Members
Points: 2823 Visits: 1406
Your way did work with a single entry in $Remove. When I added a second entry, neither was removed??
Weird.

I was also curious as to why you used a for ... i-- ...next, instead of for ... i++ ... next approach?

I didn't investigate further since I already had a working solution.

I wondered if Powershell could "remember" which values it had already looked up in the $Remove list, so that the next time it encountered one already known to be removed, it wouldn't scan the list again. (I think I read somewhere that it can use the equivalent of indexes on sorted data -- don't quote me on that).

It's taking ~4-5 secs to process the 13K records.

So, if I add the desired records to [Array]$FinalList, it takes 15 seconds to pass the array to out-file and write it to disk.
If I instead create a [string]$FinalList, assembling the string takes longer (10-11 seconds), but the out-file is 2 seconds.
And before you ask ;-), the hybrid clean into array, transform to string, and write to file takes the same total amount of time.

What I'm doing is going through AD, getting the memberships for selected groups, and then removing those users which are member of a Disabled OU. I've got this down to 26 seconds, eclipsing my best time of 10+mins using the QAD cmdlets. Since this rarely runs even once a day, I'm OK with that performance.



bruce 1565
bruce 1565
Mr or Mrs. 500
Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)Mr or Mrs. 500 (543 reputation)

Group: General Forum Members
Points: 543 Visits: 1031
26 seconds versus 10+ minutes sounds like a winner to me.
The $i—was to go backward through the $Results list since removing an item changed the indexing causing the errors and missing the duplicate “User2” entries.
I don’t think I helped you at all but appreciate your feedback.
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search