bash - Unix: “ls” command shows files with ? after the extension
2014-07
Vinothkumar
I have written a shell script (test.sh) using Java. This shell script actually does a copy job from one file to the other. After the execution of the shell script I have opened the directory from Console and typed ls. It shows the output file with ? after the extension.
example : foo.csv?
File execFile = new File(file);
FileWriter fwFile;
try {
fwFile = new FileWriter(execFile);
execFile.setExecutable(true);
BufferedWriter bwFile = new BufferedWriter(fwFile);
bwFile.write(strOutput.substring(0, 2));
bwFile.write("\r\n");
bwFile.write("cd " + strOutput);
bwFile.write("\r\n");
bwFile.write("mkdir " + strOutput);
bwFile.write("\r\n");
bwFile.write(strUnixPath);
bwFile.write("\r\n");
bwFile.write("cd " + strWorkingPath + IPlatinumConstants.FS+"lib"+IPlatinumConstants.FS+"Unx");
bwFile.write("\r\n");
bwFile.write("echo Cut Src Start time %time%");
bwFile.write("\r\n");
bwFile.write("cp " + " \"" + strSourceFilePath + "\" \""
+ strOutput + "copy_A\"");
bwFile.write("\r\n");
grawity
My guess:
Your Java program is using \r\n, carriage return + line feed, to terminate each line. However, Unix systems use only the line feed \n as a new-line indicator. While the carriage return is understood by terminals, it has no meaning at all to other programs – it's just a character like any other as far as the /bin/sh interpreter is concerned. This means that the \r becomes part of the last parameter of each command that your script runs:
cd myoutput\r
mkdir myoutput\r
cd something/lib/something\r
echo Cut Src Start time %time%\r
cp "sourcefile" "myoutputcopy_A"\r
So the last command is always creating files that have the carriage-return at the end of their name, and ls shows them as a question mark like all other "unprintable" characters. You can see this using ls -b or ls -Q.
To fix this, use only \n when generating Unix shell scripts. (Or, even better, don't generate shell scripts – instead, write scripts that accept parameters where needed. You could do many things in the Java program itself, as well.)
bguiz
I would like to append both the last command and the output of the last command to a text file for the purpose of a tutorial.
For example: After I do an ls in my home directory I see this on the screen
bguiz@sheen:~$ ls
Desktop Music Documents
I want to then be able to enter a single command which will append the following to a textfile named cmd.txt
$ ls
Desktop Music Documents
The idea is that each time I enter a command, I can log both the command itself and its output to the same file, and after several commands, it will demonstrate a particular series of commands. I know this can be done manually - but why do that if there's an easy alternative, right?
This is what I've cooked up so far:
echo -n "\$ " >> cmd.txt; echo !-1:p >> cmd.txt; !-1 >> cmd.txt
It works, but is rather clunky, and has several gotchas such as not being able to preserve the exact screen formatting.
Is the a more elegant solution?
Thank you for the answers so far, but I have a requiement that it needs to work with pipe, e.g.:
ls -lart | grep ^d
Needs to get this appended in the file:
$ ls -lart | grep ^d
drwx------ 14 bguiz staff 4096 2010-03-03 15:52 .cache
drwx------ 5 bguiz staff 4096 2010-03-03 09:38 .ssh
Duane
[script.ksh] run with "script.ksh ls"
#!/bin/ksh
OUTPUT="cmd.txt"
if [[ $# -eq 0 ]];then
print "no command to run"
exit
fi
# dump command being run to file
echo $@ >> $OUTPUT
# run command and output stdout to the screen and file
$@ | tee -a $OUTPUT
dmckee
Rather simpler than the cleverness you've been trying:
$ script cmd.txt
do what you want to document here
then hit Control-d.
You can edit the file at a later date to add annotations if you wish, and you can use
$ script -a cmd.txt
to append more text to an existing file.
The available options seem to vary considerably between implementation.
- The Mac OS X (i.e. BSD) script supports
-kwhich logs keyboard input to the command - The GNU version (found on linux systems) supports
-cwhich allows you to specify the command on the original command line, allowing you to skip the "type your demonstration here then hit control-d" bit - The BSD version can also specify the command on the command line but does not accept a flag for that instead it must follow the output filename (which is required in this case).
Finally the GNU version warns the vi is not well represented in type scripts (and I imagine that this warning applies equally well to all other commands that make use of curses).
mrucci
Write a script script.sh like the following were you insert annotation in the form of comments:
#!/bin/sh -v
# Annotating the behaviour of the ls command
ls -l
# Other comments on the next command
cmd
Note the -v switch in the first line:
-v verbose The shell writes its input to standard error as it is read.
Then execute the script redirecting both stdout and stderr to the file cmd.txt using:
$ ./script.sh > cmd.txt 2>&1
The file cmd.txt will contain the annotations, the commands and their relative output like:
# Annotating the behaviour of the ls command
ls -l
total 1824
drwxr-xr-x 11 mrucci mrucci 4096 2010-02-14 18:16 apps
drwxr-xr-x 2 mrucci mrucci 4096 2010-02-20 12:54 bin
-rw------- 1 mrucci mrucci 117469 2010-02-25 11:02 todo.txt
# Other comments on the next command
cmd
./testscript.sh: 7: cmd: not foun
PS: remember to give execution permission to the script with:
$ chmod +x script.sh
Jonathan Leffler
In the circumstances, I generally use one of two techniques:
sh -x script
Or simply run the commands and then use copy'n'paste to save the material to a file.
The 'sh -x' output doesn't include the normal PS1 prompt - it uses the PS3 which defaults to '+ ', IIRC. And it slightly modifies the output - the details vary a bit depending on the shell you are using (so 'bash' does more messing than 'ksh' or Bourne shell do). I seldom need more material than copy'n'paste can manage, so it is my predominant modus operandi for StackOverflow and SuperUser, etc.