File sharing in early versions of MTS

posted May 21, 2014, 6:43 AM by Jeff Ogden   [ updated May 25, 2014, 7:17 AM ]

E-mail to Gary asking about the stages of file sharing in MTS

From: Jeff Ogden
Subject: file sharing in early versions of MTS
Date: May 18, 2014 5:35:55 PM EDT
To: Gary Pirkola

Gary, am I remembering this early history correctly? The first of it was before my time at the CC.

(Stage 1) In the beginning:

Library (public) files are readable by all users and modifiable by people who know a special password (limited to CC staff).
User files are readable and modifiable by their owners and no one else.

(Stage 2) Later:

It became possible to "permit" user files to be readable (read-only or RO) by all other users. There was a program to do this, but I forget its name, was it *Permit? [yes, it was]

(Stage 3) And later still:

Really shared files added file locking  and the ability to permit files to ccids, project ids, and pkeys as well as initial substrings of ccids, project ids, and pkeys using the $Permit command in a wide variety of ways (read, write-change, write-expand, destroy, …).
File locking and deadlock detection was introduced.

I think that that is pretty much it. 

Were pkeys introduced at the same time as the ability to permit to ccids and project ids?

Before file locking was introduced what, if anything, protected a file that was being read by some users and modified by its owner at the same time?

I assume that needing a password to modify a library file was discontinued at stage (2) and from then on there was really nothing too different about library files and user files.

I guess I could go read your IEEE paper, but I don't have a copy at home and am too lazy to go to the U-M Library.

  -Jeff

Gary's reply

From: Gary Pirkola
Subject: Re: file sharing in early versions of MTS
Date: May 20, 2014 12:43:31 PM EDT
To: Jeff Ogden
Cc: Michael Alexander

Sorry about not answering sooner, but I wanted to go back and read a couple of MTS file system papers myself.

On May 18, 2014, at 5:35 PM, Jeff Ogden wrote:

Gary, am I remembering this early history correctly? The first of it was before my time at the CC.

(Stage 1) In the beginning:

  • Library (public) files are readable by all users and modifiable by people who know a special password (limited to CC staff).
  • User files are readable and modifiable by their owners and no one else.

I believe this is correct

(Stage 2) Later:

  • It became possible to "permit" user files to be readable (read-only or RO) by all other users. There was a program to do this, but I forget its name, was it *Permit?

My memory is not very clear about what was available/possible at this intermediate stage. Mike? Did we really have "read only" data files for example or just "run only" object files for users?

(Stage 3) And later still:

  • Really shared files added file locking and the ability to permit files to ccids, project ids, and pkeys as well as initial substrings of ccids, project ids, and pkeys using the $Permit command in a wide variety of ways (read, write-change, write-expand, destroy, …).
  • File locking and deadlock detection was introduced.

Yes, this part is correct, I remember well :)

I think that that is pretty much it.

Were pkeys introduced at the same time as the ability to permit to ccids and project ids?

Actually no. I just reread the paper John and I wrote called “Protection of Information in a General Purpose Time Sharing Environment” for a Computer Security and Integrity Conference in May, 1977, and it says that "the facility for sharing files among users and projects has been available for close to 4 years (since 1973?), whereas the facility for giving programs access to files has been available for only about one year (since 1976?)”

Before file locking was introduced what, if anything, protected a file that was being read by some users and modified by its owner at the same time?

Good question. Mike? Was there any way for the “lower level system” or higher level MTS command interface to detect and prevent this? There was nothing in the file system proper that I remember before really shared files, but I can’t image that we would have just let this happen? If we only had run only object files, this would not have been quite so serious a problem.

I assume that needing a password to modify a library file was discontinued at stage (2) and from then on there was really nothing too different about library files and user files.

I’m not sure. If we had something that worked for library files, and something else that worked for Read Only/Run Only user files, we might have just let both continue as is, until we implemented really shared files. Mike?

I guess I could go read your IEEE paper, but I don't have a copy at home and am too lazy to go to the U-M Library.

Since I did end up finding and rereading a couple of these MTS file system papers, I’m motivated again to organize and let you look at some of my CC historical archive stuff. Maybe when you’re over here for the "Baja photofest", you can see what I have and if it looks interesting, you can take a couple of boxes home for further investigation.

Gary

Some information from the U-M Computing Center Newsletter

From: Jeff Ogden
Subject: Re: file sharing in early versions of MTS
Date: May 20, 2014 7:55:14 PM EDT
To: Gary Pirkola
Cc: Michael Alexander

I found this on pages 7 and 8 of the U-M CC Newsletter, Vol. 1, No. 2, 21 January 1971:



From: Jeff Ogden
Subject: Re: file sharing in early versions of MTS
Date: May 20, 2014 11:47:44 PM EDT
To: Gary Pirkola
Cc: Michael Alexander

And a little more:

From the CC Newsletter, Vol. 2, No. 8, 7 June 1972, page 5:


From the CC Newsletter, Vol. 2, No. 9, 30 June 1972, page 6:
From the CC Newsletter, Vol. 2, No. 15, 23 October 1972, page 1:
Same article page 4:

Another reply from Gary

On May 21, 2014 10:15:05 AM -0400 Gary Pirkola wrote:

Well that certainly answers my question about what you could do with
the *permit program. Thanks Jeff. So now I’m even more intrigued to
remember how we prevented an owner from modifying a data file while
other users were reading it. Seems like a recipe for an inconsistent
file down at the file buffer level otherwise.  Gary

A reply from Mike about *PERMIT and *COPY (stage 2)

On May 24, 2014, at 8:06 PM, Mike Alexander wrote:

It's part of the answer, but not quite all of it.  I've attached two sections from the 3rd edition (November 1969) of Volume 2 for *PERMIT and *COPY.  If you're really interested, I've also attached the source for both of them from D2.  I also have extracted the source for OPEN, FCBEQU and CATEQU from D2 if you want them.

Although the writeup for *Permit talks about both a "how" and a "who" parameter, it really only takes one parameter other than the file name. If present, it must be one of "RO", "ALL", "NONE", or "PRJNO" where "ALL" is the default.  RO makes the file read only to everyone, including the owner.  "NONE", of course, makes it unlimited to the owner and no access for anyone else.  "ALL" and "PRJNO" interact with *COPY.  If either of these is given then others (either all others or in the same project) can copy the file using *COPY but not access it otherwise.  I think the owner still has write access but I didn't verify that.

Hence the snippet from the Newsletter that maps *PERMIT to Really Shared Files is a bit misleading.  "ALL" maps to U Owner, R All, but they aren't really the same since *Permit's "ALL" only allowed access by *COPY.

*PERMIT

[From MTS Volume 2, Public File Descriptions, Third Edition, November 1969.]

 Contents:      The object module of a program to allow users to permit their
                files to be accessed by other users.

 Purpose:       To allow files to be shared between user ID numbers.

 Usage:         The program is invoked by a $RUN command.

 Logical I/O Units Referenced:
                GUSER  - filename, access type and/or sharers
                          (unless parameters are given).

 Description:   Three parameters separated by blanks are  required  as  input
                for each file being permitted.  They are:

                          What  How  Who

                What    = the name of the file to be shared.

                How     = the  type  of  sharing  access  allowed.  Currently
                          Read-Only is the only type of access allowed and is
                          designated by RO.  This parameter should be omitted
                          if Read-Only is not desired.

                          Notes:  1.  Designation of a  file  by  a  user  as
                                  Read-Only  shared  allows  other  users  to
                                  access that file directly  (without  having
                                  to  copy  it  via the *COPY program).  This
                                  file may then only be read - it may not  be
                                  changed, even by the owner of the file.
                                  2.    If  the  owner  of  the  file "X" has
                                  SIGNON ID ABCD, then  the  filename  ABCD:X
                                  will  refer  to  this  file and may be used
                                  anywhere a filename may  be  used  provided
                                  that  the  usage  is  a read-type usage and
                                  provided that user ABCD has permitted  file
                                  "X" as Read-Only shared.
                                  3.    If  one  has  a file whose name has a
                                  colon as one of the first five  characters,
                                  he  must  refer  to  it using XXXX:filename
                                  where XXXX is his SIGNON ID ,e.g., if  user
                                  ABCD  has  file  0:1 he must refer to it as
                                  ABCD:0:1.
                                  4.   The file should be completely prepared
                                  and checked before being permitted  as  RO.
                                  If  changes  must be made later, it must be
                                  unpermitted (permitted  with  NONE)  before
                                  the  changes can be made.  This must not be
                                  done when someone else is using  it.   Pre-
                                  sently  there is no check made to determine
                                  whether someone is accessing the file  when
                                  it is being unpermitted.

                Who     = the  sharers  who  are  allowed to access the file.
                          Currently three classes  of  sharers  are  allowed.
                          These are:
                          ALL     - meaning all users (default).
                          PRJNO   - meaning  all  users  who  have  the  same
                                    project number as  the  permitting  user.
                                    The  project  number  appears  as "charge
                                    number" on the tail-sheet of batch  runs.
                                    It is not (in most cases) the same as the
                                    SIGNON ID.  Students in a class, or other
                                    groupings  of  users,  are  assigned dif-
                                    ferent SIGNON ID's but the  same  project
                                    number.
                          NONE    - meaning no one else.

 Notes:                             1.  Current restriction:  If RO is speci-
                                    fied, ALL is assumed.
                                    2.   If  PRJNO  or  ALL is specified, the
                                    program *COPY may be run to make  a  copy
                                    of the file under a new user ID.
                                    3.  When a file is created, it has access
                                    type NONE.

 Examples:      $RUN *PERMIT  PAR=MYFILE RO

                    permits the file to be Read-Only shared by all users.

                $RUN *PERMIT  PAR=MYFILE2

                    permits the file to be copied by all users via *COPY.

                   To  permit  several  files,  the parameter field should be
                omitted.   The  program  will  request  the  information  via
                logical I/O unit GUSER.

                In  either  terminal  or  batch  mode,  the program will read
                lines, each having a file  name,  an  access  type  and/or  a
                sharing class, until an end of file is read.

*COPY

[From MTS Volume 2, Public File Descriptions, Third Edition, November 1969.]
 
 Contents:     The  object  module of a program which copies permitted files
               (line or sequential) from one user account to another.

 Usage:        The program is invoked by a $RUN command specifying *COPY  as
               the location of the object module.

 Prototype:    $RUN  *COPY  [SPUNCH=FDname  PAR=userid:FDname]

 Examples:     To copy one file:
                    $RUN  *COPY  SPUNCH=-T  PAR=STAT:NEWS

               To copy a series of files (user input is in lower case):
                    $run *copy
                    EXECUTION BEGINS
                    ENTER OUTPUT FDNAME
                    -x
                    ENTER "SIGNON:FILE-NAME"
                    stat:news
                    ENTER OUTPUT FDNAME
                    -y
                    ENTER "SIGNON:FILE-NAME"
                    xxxx:hisfile
                    INPUT FILE NOT FOUND
                    $endfile
                    EXECUTION TERMINATED

 Logical I/O Units Referenced:
               SPUNCH - Output of the files being copied.
               GUSER  - Input of "to" and "from" file names if no PAR= field
                        was specified in the $RUN command.

 Description:  To  copy  one  file,  the  file  being  copied from should be
               specified in the PAR= field of the $RUN command, and the file
               to be copied into should be  attached  to  logical  I/O  unit
               SPUNCH.  The "from" file should be specified by putting the 4
               character  signon  ID followed by a colon(:)  before the file
               name.  For example, to copy HISFILE under the  ID  XXXX  into
               -MYCOPY use the command
                    $RUN *COPY SPUNCH=-MYCOPY PAR=XXXX:HISFILE
               An  error  comment  will  be  produced if access to the other
               user's file is not permitted.

               To copy several files, the PAR= field should be omitted.  The
               program will request, via GUSER, two  lines  for  each  copy:
               the  first is the name of the "to" file and the second is the
               name of the "from" file.  As in  *PERMIT,  the  program  will
               read until an end-of-file is reached.

               The  "from"  file,  if  it  is  a line file, may specify line
               number ranges, as for  example,  XXXX:HISFILE(1000,5000)  but
               may  not specify explicit concatenation.  Any implicit conca-
               tenation specified in the "from" file will be ignored.

               Line files may be copied into sequential or  sequential  with
               line  number  files  and vice-versa.  It should be noted that
               line number ranges and the  @I  modifier  on  the  "to"  file
               operate exactly as they would on an MTS $COPY command.

               Presently  *COPY  is  the  only way files permitted to ALL or
               PRJNO may be copied.  Files permitted RO may be copied either
               by $COPY or *COPY.

A summary and more questions about stage 2 from Jeff

From: Jeff Ogden
Subject: Re: file sharing in early versions of MTS
Date: May 24, 2014 11:15:11 PM EDT
To: Mike Alexander
Cc: Gary Pirkola

Thanks Mike. This is interesting. I vaguely remembered *PERMIT and RO files. I didn't know about or forgot about *COPY.

I haven't looked at the source yet, but as far as managing simultaneous access to shared files, it seems as if there were two approaches:

1) Make the files RO to everyone including the owner so that the issue of simultaneous changes to files that others have open cannot occur or at least not unless you un-permit the file using *PERMIT and in that case we just cross our fingers and hope that things work out. And at least for fairly small files where the entire file will fit into a task's file buffers, things should work out OK.

2) If the files are permitted, but not RO, then let *COPY worry about the simultaneous access issues.

I'm going down this path partly because I'm interested and partly because the "MTS System Architecture" article in Wikipedia doesn't currently talk about shared files or file locking and so I'm gathering information for an addition to that article.

   -Jeff

A reply about simultaneous access during stage 2 from Mike

On May 25, 2014, at 1:05 AM, Mike Alexander wrote:

It's not quite that good.  *COPY didn't worry about simultaneous access issues, it just copied the file and hoped it happened quickly enough that it got something consistent.  At least the original file couldn't be damaged by this, even if the copy was corrupted.  In practice I never heard of a case in which this caused problems.

          Mike


Comments