[Discuss] Any Subversion geniuses out there?

[matt at mattshields.org (Matt Shields)]

On Fri, Dec 2, 2011 at 10:01 AM, John Abreau <abreauj at gmail.com> wrote:

> I've seen this before with "text" files on Windows. Just changing the
> MIME type wil not work, because the files are encoded in UTF-16
> (note *NOT* UTF-8). 16-bit characters, not 8-bit characters. If you
> change the MIME type to force it to be interpreted as normal text,
> the file will have a null byte between each and every character.
>
> When I had to deal with those issues at a previous job, I used iconv(1)
> in my shell scripts to convert the MS "text" to UTF-8.
>
>    iconv --from-code=UTF-16 --to-code=UTF-8 ms-text-file.txt >
> plain-text-file.txt
>
> I also ran it through "tr -d '\r'" to scrape off the ^M at the end of
> each line before dropping it into the output file, but that's a separate
> issue.
>
>
> On Fri, Dec 2, 2011 at 9:40 AM, Matt Shields <matt at mattshields.org> wrote:
> > On Fri, Dec 2, 2011 at 8:11 AM, Edward Ned Harvey <blu at nedharvey.com>
> wrote:
> >
> >> > From: discuss-bounces+blu=nedharvey.com at blu.org [mailto:discuss-
> >> > bounces+blu=nedharvey.com at blu.org] On Behalf Of Matt Shields
> >> >
> >> >  What I was wondering is it possible in Subversion when a changeset is
> >> > being committed that a hook could be used to change the mime-type.
>  So if
> >> > the file being committed is a *.sql, then it would override whatever
> >> > mime-type the client is saying and apply text/x-sql.
> >>
> >> This question will be best answered by the subversion-users mailing
> list,
> >> http://subversion.apache.org/mailing-lists.html
> >> but let's see what we can say about it here.
> >>
> >> The mime type, I believe, is determined by the svn client, and it's
> >> determined by file contents.  What do you get, if you run linux "file"
> on
> >> the file?  What do you see if you try to open the file in vim or emacs?
> >>
> >> I'm sure you can change the mime-type as a precommit or postcommit hook
> >> (probably best precommit) but I'm almost equally sure that it's not what
> >> you
> >> want to do.  When they detect the contents and select a mime type, the
> >> reason they're doing it is because svn internally employs all sorts of
> diff
> >> and compression algorithms, to optimize both the network traffic and
> disk
> >> storage.  If you go overriding the mime types against its natural
> wishes,
> >> you run the risk of ...  Suboptimizing performance.  Is probably the
> >> diplomatic way of saying effing everything up.
> >>
> >> Another option you might consider, I believe, is that they have a
> mechanism
> >> of some kind to allow you to inject a custom client-side diff utility
> for
> >> certain files or mime types or something like that.  You might
> configure it
> >> so that your client doing the diff might run something like the SQL
> >> equivalent of "dos2unix" to convert a file format and then diff it, or
> >> something like that.  Of course the odds of success doing this are
> >> diminished by trac.  You might just have to use something like
> tortoisesvn
> >> or whatever to perform these diffs.
> >>
> >> In fact, tortoisesvn does some pretty excellent diffing.  What happens
> if
> >> you try diffing with tortoise?
> >>
> >>
> > Yes, I'm aware of that, and I can put something in each client's
> svnconfig
> > to override this behavior for specific filetypes.  I don't want to have
> to
> > do that since everytime we get a new developer it's one more step I have
> to
> > remember to do to their dev machine.
> >
> > The issue is SQL Server Management Studio is encoding it weird and
> > TortoiseSVN is then taking that as it being a binary and not a text file.
> >  See the two outputs of file.  The first has been fixed by me forcing it
> to
> > be proper encoding and the proper mime-type.  The second was created in
> > SSMS and committed.
> >
> > dbo.Proc_xxxx.sql:         Little-endian UTF-16 Unicode c program text,
> > with CRLF, CR line terminators
> > dbo.Proc_yyyy.sql:                 ASCII c program text, with CRLF line
> > terminators
> >
> > Yes, diff's in TortoiseSVN are great, same with Unix command line.  The
> > issue is the Dir of Tech prefer's to use Trac to review all changes, and
> > because it's encoded wrong, that means svn is applying the wrong
> mime-type
> > which causes Trac's diff feature not to work.
> >
> > In this case I don't believe there is any harm forcing svn to use a
> > specific mime-type since they are both text. I'll check out the
> > check-mime-type.pl that Greg mentioned.
> >
> > Matthew Shields
> > Owner
> > BeanTown Host - Web Hosting, Domain Names, Dedicated Servers, Colocation,
> > Managed Services
> > www.beantownhost.com
> > www.sysadminvalley.com
> > www.jeeprally.com
> > Like us on Facebook <http://www.facebook.com/beantownhost>
> > Follow us on Twitter <https://twitter.com/#!/beantownhost>
> > _______________________________________________
> > Discuss mailing list
> > Discuss at blu.org
> > http://lists.blu.org/mailman/listinfo/discuss
>
>
>
> --
> John Abreau / Executive Director, Boston Linux & Unix
> OLD GnuPG KeyID: D5C7B5D9 / Email: abreauj at gmail.com
> OLD GnuPG FP: 72 FB 39 4F 3C 3B D6 5B E0 C8 5A 6E F1 2C BE 99
> 2011 PGP KeyID: 32A492D8 / Email: abreauj at gmail.com
> 2011 PGP FP: 7834 AEC2 EFA3 565C A4B6  9BA4 0ACB AD85 32A4 92D8
>

That sounds like I could do that as a one time, then recommit all the files
that have been processed.  But going forward I would still need a precommit
script to do this

How about having Trac ignore the svn:mime-type when it determines if it can
do a diff or not?  I'll have to warn the dev's and Director of Tech not to
do a diff on any real binary files


Matthew Shields
Owner
BeanTown Host - Web Hosting, Domain Names, Dedicated Servers, Colocation,
Managed Services
www.beantownhost.com
www.sysadminvalley.com
www.jeeprally.com
Like us on Facebook <http://www.facebook.com/beantownhost>
Follow us on Twitter <https://twitter.com/#!/beantownhost>