| Author |
Topic  |
|
|
Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline |
Posted - 06/03/2011 : 12:48:58 PM
|
Here's an odd request that I'm trying to script: Task: move all files located in spreadsheet to a remote server for possible deletion.
I have a spreadsheet containing thousands of files scattered across multiple servers that need to be moved to a folder located on a single server. The spreadsheet contains the full UNC of the file(s) on the source server. What I'd like is for the script to recreate the same UNC on the destination server and move the file into it. Maintaining the ACL's on the files would alse be nice but isn't necessary as the main goal at this point is to move the files from the designated locations and delete them shortly after. If necessary I can always rollback by manually copying the file back to its original location. My searches are fubar as I'm only getting hits on the import-csv cmdlet. Any responses are direction appreciated.
|
|
|
Nobody
Here To Stay
 
USA
184 Posts
Status: offline |
Posted - 06/03/2011 : 1:22:23 PM
|
| Can you post some examples of the spreadsheet format, and the source/destination paths? |
aka - Matt www.SnowTrek.org |
 |
|
|
Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline |
Posted - 06/03/2011 : 2:20:53 PM
|
Standard UNC path: \\Servername\d$\folder\filename.txt
I'd need to recreate the UNC path on the destination server instead of dumping everything into a folder.
Tia |
Edited by - Phil Marcum on 06/03/2011 2:24:52 PM |
 |
|
|
cj_berlin
Honorable But Hopeless Addict
    
Germany
3964 Posts
Status: offline |
Posted - 06/03/2011 : 3:09:17 PM
|
Phil,
by 'spreadsheet', do you mean an Excel spreadsheet? |
Evgenij Smirnov
|
 |
|
|
Nobody
Here To Stay
 
USA
184 Posts
Status: offline |
Posted - 06/03/2011 : 4:01:55 PM
|
| Really need to know a couple specifics before we can help script anything. As asked above, is the spreadsheet just a one column excel spreadsheet with a unc path in each row? Are the files on the D$ share of each server? Are the folder/file names following some nameing convention? Are there going to be duplicate named folders/files on each server? How deep might the folder structure go? etc etc etc. So basically a couple examples would clear up most of the questions I have. |
aka - Matt www.SnowTrek.org |
 |
|
|
cj_berlin
Honorable But Hopeless Addict
    
Germany
3964 Posts
Status: offline |
Posted - 06/03/2011 : 4:10:56 PM
|
quote: Originally posted by Nobody
How deep might the folder structure go?
This is an important one. You would be dealing with functions that expect the path as string so if those strings can potentially grow longer than 255 character, you'd need to use unicode functions which is a (minor) PITA... |
Evgenij Smirnov
|
 |
|
|
Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline |
Posted - 06/03/2011 : 5:03:10 PM
|
Basically This is the format of the paths throughout the spreadsheet: \\ServernameA\d$\folder\filename.txt \\ServernameZ\e$\folder\folder\folder\filename1.xls \\ServernameZ\e$\folder\folder\folder\filename2.xls \\ServernameD\g$\folder\folder\folder\folder\filename2.xls
1. Yes I'm dealing with an Excel spreadsheet which has all of the paths listed in one row / column.
2. The paths are to different servers and different shares where some go as far as 4 or 5 folders deep. I'd need to confirm if any extend beyond 255 characters.
3. There are multiple files in separate locations on the same server. So one server may have 5 files in different folders and another may have 2 files, etc. So I may have to sort the spreadsheet accordingly and go from there. The less intervention the better. Hopefully I didn't miss anything.
tia |
 |
|
|
cj_berlin
Honorable But Hopeless Addict
    
Germany
3964 Posts
Status: offline |
Posted - 06/03/2011 : 5:06:43 PM
|
OK, so here's a rough but working draft of an Excel macro. You'd have to reference the Windows Scripting Host library in your project to make this work.
Sub MoveThemAll()
Dim strSrcBase As String
Dim strDestBase As String
Dim fsoX As New FileSystemObject
Dim i, intLastLine As Integer
Dim strDestPath, strFile, strXPath As String
strSrcBase = "\\OLDSERVER\COMMONSHARE_IF_ANY\" 'with a trailing backslash
strDestBase = "\\NEWSERVER\COMMONSHARE_IF_ANY\" 'with a trailing backslash
intLastLine = 3 'number of significant line. This macro assumes that the file paths are in column A starting with row 1 and will write status output into columns B (for copy and/or create dest folder) and C (for delete)
For i = 1 To intLastLine
strXPath = Range("A" & Trim(Str(i))).Formula
If fsoX.FileExists(strXPath) Then
If Left(UCase(strXPath), Len(strSrcBase)) = UCase(strSrcBase) Then
strFile = fsoX.GetFileName(strXPath)
strDestPath = fsoX.GetParentFolderName(strXPath) & "\"
strDestPath = Replace(UCase(strDestPath), UCase(strSrcBase), UCase(strDestBase))
If Not fsoX.FolderExists(strDestPath) Then
MakeDestFolder (strDestPath)
End If
If fsoX.FolderExists(strDestPath) Then
fsoX.CopyFile strXPath, strDestPath & strFile
If fsoX.FileExists(strDestPath & strFile) Then
Range("B" & Trim(Str(i))).Formula = "copied successfuly"
fsoX.DeleteFile strXPath
If fsoX.FileExists(strXPath) Then
Range("C" & Trim(Str(i))).Formula = "delete failed"
Else
Range("C" & Trim(Str(i))).Formula = "deleted successfuly"
End If
Else
Range("B" & Trim(Str(i))).Formula = "copy failed"
End If
Else
Range("B" & Trim(Str(i))).Formula = "Don't seem to be able to create " & strDestPath
End If
End If
End If
Next i
End Sub
Sub MakeDestFolder(ByVal xPath As String)
Dim fsoX As New FileSystemObject
Dim xPF As String
xPF = fsoX.GetParentFolderName(xPath)
If fsoX.FolderExists(xPF) Then
fsoX.CreateFolder xPath
Else
MakeDestFolder (xPF)
If fsoX.FolderExists(xPF) Then fsoX.CreateFolder xPath
End If
End Sub
You'll have to rerun it for each source server.
EDIT: If ACLs are of any value for you, I'd have the VBA invoke a robocopy command. |
Evgenij Smirnov
|
Edited by - cj_berlin on 06/03/2011 5:10:39 PM |
 |
|
|
Nobody
Here To Stay
 
USA
184 Posts
Status: offline |
Posted - 06/06/2011 : 11:30:06 AM
|
This is what I came up with. I didn't do any error handling or logging, but the basics seem to work. Requires that you save your spreadsheet as a .txt file.
Powershell
$NewServer = '\\NewServer\Share' # Your New Server and Share
$Files = 'files.txt' # Save Excel file to text file
##################################################################
$arryUNC = @(get-content $files) # Loading each UNC path to an array
Foreach ($i in $arryUNC){ # Loop through the array
[string]$leaf = [regex]::split($i,"^\\\\\w+?(?:\\[\w\s$]+)") # Split file path from server\share
$leaf = $leaf -replace " ","" # trim that pesky leading space
$destination = $NewServer + $leaf # Set destination path variable
echo "f"|xcopy $i $destination /I /O /Y # Using xcopy to copy file. Preserves ACL
}
|
aka - Matt www.SnowTrek.org |
Edited by - Nobody on 06/06/2011 11:42:43 AM |
 |
|
|
Xenophane
Honorable But Hopeless Addict
    
Denmark
3070 Posts
Status: offline |
Posted - 06/06/2011 : 6:06:00 PM
|
Just made a little update to Nobody's script:
$paths = @()
$paths = import-csv C:\temp\UNCPaths.csv
$newServer = "\\apt410-ctn\c$\Scripts"
Foreach ($path in $paths) {
$NewPath = "$newServer$($path.File.substring(1))"
$($path.File)
$NewPath
Write-Output "f" | xcopy $($path.File) $NewPath /I /O /Y
#Catch { Write-Output "Error: Xcopy: $_">>c:\temp\Failed.log}
If ($?) {Write-Output "Success: Xcopy $($path.File) -> $NewPath ">>c:\temp\Succeeded.log }
else {Write-Output "Error: Xcopy: $($path.File)">>c:\temp\Failed.log}
}
It will write to files, one with errors and the other with successfull copies.
In this script I use the old path, to create the folder structure on the new server, to make sure, I do not get any problems with duplicate filenames... |
Microsoft Powershell MVP
SIG> George Bernard Shaw : The power of accurate observation is commonly called cynicism by those who have not got it. </SIG>
You can read my blog at www.xipher.dk |
 |
|
|
Nobody
Here To Stay
 
USA
184 Posts
Status: offline |
Posted - 06/07/2011 : 1:07:20 PM
|
Xenophane, I was really struggling with your version for a bit... I didn't have the column header in my csv file 
I'm still a powershell noob, Can you explain:
$($path.File) vs $path.File |
aka - Matt www.SnowTrek.org |
 |
|
|
Xenophane
Honorable But Hopeless Addict
    
Denmark
3070 Posts
Status: offline |
Posted - 06/07/2011 : 3:38:11 PM
|
Sorry about that I wanted to post it, but my 7 month old woke up, so after lulling him back to sleep I completely forgot :)
Below is my .csv file quote:
File \\localhost\c$\temp\123.txt \\localhost\c$\temp\1234.txt \\apt410-ctn\c$\temp\12345.txt
$($path.File) vs $path.File It should not be necessary in this example I more or less do it as a habit.. When you wrap you variable in $() it gets treated like an expression and evaluated. Example: (Red is what is output to the console)
PS C:\> notepad
PS C:\> Get-Process notepad
Handles NPM(K) PM(K) WS(K) VM(M) CPU(s) Id ProcessName
------- ------ ----- ----- ----- ------ -- -----------
70 7 1944 7128 79 0,05 8740 notepad
PS C:\> $note = Get-Process notepad
PS C:\> Write-host "Calc Uses $note.handles handles"
Calc Uses System.Diagnostics.Process (notepad).handles handles
PS C:\> write-host "Calc uses $($note.handles) handles"
Calc uses 70 handles
PS C:\>
|
Microsoft Powershell MVP
SIG> George Bernard Shaw : The power of accurate observation is commonly called cynicism by those who have not got it. </SIG>
You can read my blog at www.xipher.dk |
 |
|
|
Nobody
Here To Stay
 
USA
184 Posts
Status: offline |
Posted - 06/07/2011 : 4:26:33 PM
|
Thanks for clearing that up. That was the solution to a problem I had recently. I had to work around it like this...
$note = Get-Process notepad|foreach-object {$_.handles}
write-host "Notepad uses $note handles"
heh...I've met my learning quota for the week, and it's only Tuesday! Thanks. |
aka - Matt www.SnowTrek.org |
 |
|
|
Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline |
Posted - 06/22/2011 : 3:50:31 PM
|
Almost a month later and I'm just getting the green light to move forward with this project. Being that I need to move the files from one location to another server creating the same path I'm going to give "Nobody's" script a try. Here's my dilema: 1. I won't say that the script doesn't work, I just can't get it to work for me. Nobody's original powershell script called for the spreadsheet to be saved as a text file. From what I can tell Xenophane made some changes and now appears to import the .csv, spreadsheet, file instead of the text file. I've modified the following:
$paths = @() $paths = import-csv C:\temp\MySpreadsheetHere.csv $newServer = "\\MyServerName\c$\Foldername"
Foreach ($path in $paths) { $NewPath = "$newServer$($path.File.substring(1))" $($path.File) $NewPath Write-Output "f" | xcopy $($path.File) $NewPath /I /O /Y #Catch { Write-Output "Error: Xcopy: $_">>c:\temp\Failed.log}
If ($?) {Write-Output "Success: Xcopy $($path.File) -> $NewPath ">>c:\temp\Succeeded.log } else {Write-Output "Error: Xcopy: $($path.File)">>c:\temp\Failed.log} }
Save my script as filemove.ps Launch Powershell and navigate to my script:
Now I attempt to run the script: ps> movefile.ps
I get nothing. Am I doing something wrong?
tia
|
 |
|
|
Xenophane
Honorable But Hopeless Addict
    
Denmark
3070 Posts
Status: offline |
Posted - 06/23/2011 : 02:16:00 AM
|
Phil, The line containing the file paths in the csv file, has to contain the word "file" (Columnheader without the quotes) in the first line.
$paths = @()
$paths = import-csv C:\temp\MySpreadsheetHere.csv
$newServer = "\\MyServerName\c$\Foldername"
Foreach ($path in $paths) {
$NewPath = "$newServer$($path.File.substring(1))"
$($path.File)
$NewPath
}
You can just try to copy and paste the above into the console to see if you get anything (It just prints the path to the screen) |
Microsoft Powershell MVP
SIG> George Bernard Shaw : The power of accurate observation is commonly called cynicism by those who have not got it. </SIG>
You can read my blog at www.xipher.dk |
 |
|
|
Nobody
Here To Stay
 
USA
184 Posts
Status: offline |
Posted - 06/23/2011 : 11:57:34 AM
|
As mentioned, my method requires a text file with one path per line, no column headers or any extra content. You can just "save as" .txt from excel, and make sure any column headers are removed.
Xenophane's method requires a csv file with a column header named 'file'.
|
aka - Matt www.SnowTrek.org |
 |
|
|
Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline |
Posted - 06/24/2011 : 11:08:09 AM
|
Got it! Tested and confirmed that the script will copy the complete URL path and file from the source server(s) to the destination server. What I need to do now is modify it to "move" the files from the source to the destination. Tried unsuccessfully at switching the Xcopy commands with the Move command.
Any pointers? |
 |
|
|
Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline |
Posted - 06/24/2011 : 11:14:05 AM
|
Got it and able to confirm that the complete folder and file structure is copied to the destination server. What I need to do now is modify the script to move the files to the destination server. Swapping to xcopy command with move isn't working. Am I missing something?
Tia |
 |
|
|
Nobody
Here To Stay
 
USA
184 Posts
Status: offline |
Posted - 06/24/2011 : 11:46:16 AM
|
A move is nothing more than a copy and a delete. So you can just add in a Remove-Item line immediately after the xcopy, or after all files have copied, run through the foreach loop again and delete. Just be careful, if don't do any error handling, and xcopy goes sideways, it could delete all your files before they are copied.
Using Xenophane's script you can delete right after the success log entry, the log then delete if successful.
|
aka - Matt www.SnowTrek.org |
Edited by - Nobody on 06/24/2011 11:47:15 AM |
 |
|
|
Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline |
Posted - 06/24/2011 : 3:18:53 PM
|
| Stuck on the remove-item cmdlet as I need to ensure that the entire UNC path and file is copied to the destination server, which is covered. The other side is to remove the file and not the entire path from the source server as there are other files in that folder. |
 |
|
|
Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline |
Posted - 06/24/2011 : 3:22:35 PM
|
Also looking at the "move-item" cmdlet as it seems to ensure that the entire UNC path and file is moved to the destination server which is what I want.
|
Edited by - Phil Marcum on 06/24/2011 3:51:02 PM |
 |
|
|
Nobody
Here To Stay
 
USA
184 Posts
Status: offline |
Posted - 06/24/2011 : 4:21:58 PM
|
I used xcopy because that's the only tool I could think of that would replicate your folder structure and keep the ACL as well as the various other parameters that xcopy offers. Oddly, powershell doesn't seem to have a tool on par with xcopy.
"Remove-Item path/to/somefile.txt" will only delete the file specified, not the entire folder structure. Play around with these commands and build some confidence in what they do.
use the -full parameter to get detailed command info and examples
Get-Help Remove-Item -full
|
aka - Matt www.SnowTrek.org |
 |
|
|
Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline |
Posted - 07/07/2011 : 01:18:53 AM
|
No luck in getting this to copy the files and delete them using a single script and will try using a second script once the files have successfully copied. In looking at the script applied I wonder if it's possible to modify it to delete the same file that was copied earlier.
$paths = @() $paths = import-csv C:\temp\MySpreadsheetHere.csv $newServer = "\\MyServerName\c$\Foldername"
Foreach ($path in $paths) { $NewPath = "$newServer$($path.File.substring(1))" $($path.File) $NewPath Write-Output "f" | remove-item $($path.File) $NewPath /I /O /Y #Catch { Write-Output "Error: remove-item: $_">>c:\temp\Moved.log}
If ($?) {Write-Output "Success: remove-item $($path.File) -> $NewPath ">>c:\temp\Succeeded.log } else {Write-Output "Error: remove-item: $($path.File)">>c:\temp\Failed.log} }
Having it to write to a a separate would be nice for verification purposes. Am I missing something here?
Any responses appreciated. |
 |
|
|
Xenophane
Honorable But Hopeless Addict
    
Denmark
3070 Posts
Status: offline |
Posted - 07/07/2011 : 06:40:59 AM
|
| BE carefull running the above script.. You are using remove-item instead of move-item |
Microsoft Powershell MVP
SIG> George Bernard Shaw : The power of accurate observation is commonly called cynicism by those who have not got it. </SIG>
You can read my blog at www.xipher.dk |
 |
|
|
Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline |
Posted - 07/18/2011 : 10:12:32 AM
|
Now that I've successfully copied the files I need to go in and delete each of them. I could use some help figuring out how this would work with the current script. Meaning there shouldn't be a "newserver" statement:
$newServer = "\\MyServerName\c$\Foldername"
When removed it throws all sorts of errors. It looks as though I need to remove every "NewServer" reference?
Am I missing something? |
Edited by - Phil Marcum on 07/18/2011 10:30:05 AM |
 |
|
|
Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline |
Posted - 07/18/2011 : 1:06:11 PM
|
Error(s) thrown:
Remove-Item : A positional parameter cannot be found that accepts argument '\\myfilesrv3\e$\filemove\myserver\d$\myfiles\samplefile1.txt'. At D:\Projects\data\Delete\dlpdelete.ps1:9 char:31 + Write-Output "f" | remove-item <<<< $($path.File) $NewPath /I /O /Y + CategoryInfo : InvalidArgument: (:) [Remove-Item], ParameterBindingException + FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.RemoveItemCommand
Error is thrown on each file. |
 |
|
|
Xenophane
Honorable But Hopeless Addict
    
Denmark
3070 Posts
Status: offline |
Posted - 07/18/2011 : 3:38:07 PM
|
It seems to me that you are trying to pass multiple parameters to remove-item Something like this should be able to do it, remove -whatif when you are sure you really really want to do it :)
$paths = @() $paths = import-csv C:\temp\MySpreadsheetHere.csv Foreach ($path in $paths) { $($path.File) remove-item $($path.File) -force -whatif }
|
Microsoft Powershell MVP
SIG> George Bernard Shaw : The power of accurate observation is commonly called cynicism by those who have not got it. </SIG>
You can read my blog at www.xipher.dk |
 |
|
|
Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline |
Posted - 07/19/2011 : 10:55:31 AM
|
| Freakin' AWESOME!!!! |
 |
|
| |
Topic  |
|