1. A method for decoding video data, the method comprising: decoding a duration between a coded picture buffer (CPB) removal time of a first decoding unit (DU) in an access unit (AU) and a CPB removal time of a second DU, wherein the second DU is subsequent to the first DU in decoding order and in the same AU as the first DU; determining the CPB removal time of the first DU based at least in part on the decoded duration; and decoding video data of the first DU based at least in part on the CPB removal time.

2. The method of claim 1, wherein the second DU is immediately subsequent to the first DU in the AU in decoding order.

3. The method of claim 1, wherein the second DU is a last DU in the AU in decoding order.

4. The method of claim 1, further comprising: decoding one or more sub-picture level CPB parameters, wherein determining the CPB removal time of the first DU comprises determining the CPB removal time of the first DU based at least in part on the decoded duration and the sub-picture level CPB parameters.

5. The method of claim 4, wherein decoding one or more sub-picture level CPB parameters comprises: decoding a sub-picture timing supplemental enhancement information (SEI) message that is associated with the first DU.

6. The method of claim 5, wherein the second DU is a last DU in the AU in decoding order, and wherein decoding the sub-picture SEI message comprises: decoding the duration between a removal time of the last DU and the CPB removal time of the first DU in the sub-picture timing SEI message.

7. The method of claim 4, further comprising: decoding a sequence level flag to indicate presence of the sub-picture level CPB parameters either in picture timing SEI messages or in sub-picture timing SEl messages.

8. The method of claim 1, wherein the AU has a Temporalld equal to 0, the method further comprising: decoding at least one of a buffering period supplemental enhancement information (SEI) message or a recovery point SEI message associated with the AU.

9. The method of claim 1, wherein the first DU comprises a non-video coding layer (VCL) network abstraction layer (NAL) unit with nal_unit_type equal to UNSPECO, EOS_NUT, EOB_NUT, in the range of RSV_NVCL44 to RSV_NVCL47 or in the range of UNSPEC48 to UNSPEC63

10. The method of claim 1, further comprising: deriving at least one of a CPB arrival time and a CPB nominal removal time for the AU at both an access unit level and a sub-picture level regardless of a value of a syntax element that defines whether the first DU is the AU

11. The method of claim 1, further comprising: decoding one or more picture level CPB parameters, wherein determining the CPB removal time of the first DU comprises determining the CPB removal time of the first DU based at least in part on the decoded duration and the picture level CPB parameters

12. The method of claim 11, wherein decoding one or more picture level CPB parameters comprises: decoding a picture timing supplemental enhancement information (SEI) message that is associated with the first DU

13. A video decoding device comprising a video decoder configured to: decode a duration between a coded picture buffer (CPB) removal time of a first decoding unit (DU) in an access unit (AU) and a CPB removal time of a second DU, wherein the second DU is subsequent to the first DU in decoding order and in the same AU as the first DU; determine the CPB removal time of the first DU based at least in part on the decoded duration; and decode video data of the first DU based at least in part on the CPB removal time

14. The video decoding device of claim 13, wherein the second DU is immediately subsequent to the first DU in the AU in decoding order

15. The video decoding device of claim 13, wherein the second DU is a last DU in the AU in decoding order

16. The video decoding device of claim 13, the video decoder further configured to decode one or more sub-picture level CPB parameters, wherein determine the removal time of the first DU comprises determine the removal time of the first DU based at least in part on the decoded duration and the sub-picture level CPB parameters

17. The video decoding device of claim 16, wherein the video decoder is further configured to decode one or more sub-picture level CPB parameters by decoding a sub picture timing supplemental enhancement information (SEI) message that is associated with the first DU

18. The video decoding device of claim 17, wherein the second DU is a last DU in the AU in decoding order, and wherein decode the sub-picture SEI message comprises decode the duration between a removal time of the last DU and the removal time of the first DU in the sub-picture timing SEI message

19. The video decoding device of claim 16, the video decoder further configured to decode a sequence level flag to indicate presence of the sub-picture level CPB parameters either in picture timing SEI messages or in sub-picture timing SEI messages.

20. The video decoding device of claim 13, wherein the AU has a Temporalld equal to 0, the video decoder further configured to decode at least one of a buffering period supplemental enhancement information (SEI) message or a recovery point SEI message associated with the AU.

21. , The video decoding device of claim 13, wherein the first DU comprises a non video coding layer (VCL) network abstraction layer (NAL) unit with nal_unit_type equal to UNSPECO, EOS_NUT, EOBJVUT, in the range of RSV_NVCL44 to RSV_NVCL47 or in the range of UNSPEC48 to UNSPEC63.

22. , The video decoding device of claim 13, the video decoder further configured to derive at least one of a CPB arrival time and a CPB nominal removal time for the AU at both an access unit level and a sub-picture level regardless of a value of a syntax element that defines whether the first DU is the AU.

23. The video decoding device of claim 13, the video decoder further configured to decode one or more picture level CPB parameters, wherein determine the removal time of the first DU comprises determine the removal time of the first DU based at least in part on the decoded duration and the picture level CPB parameters.

24. The video decoding device of claim 23, wherein the video decoder is further configured to decode one or more picture level CPB parameters by decoding a picture timing supplemental enhancement information (SEI) message that is associated with the first DU.

25. A computer-readable storage medium having stored thereon instructions that, when executed, cause a processor of a device for decoding video data to: decode a duration between a coded picture buffer (CPB) removal time of a first decoding unit (DU) in an access unit (AU) and a CPB removal time of a second DU, wherein the second DU is subsequent to the first DU in decoding order and in the same AU as the first DU; determine the CPB removal time of the first DU based at least in part on the decoded duration; and decode video data of the first DU based at least in part on the CPB removal ti e.

26. The computer-readable storage medium of claim 25, wherein the second DU is immediately subsequent to the first DU in the AU in decoding order.

27. The computer-readable storage medium of claim 25, wherein the second DU is a last DU in the AU in decoding order.

28. The computer-readable storage medium of claim 25, wherein the instructions further cause the processor to decode one or more sub-picture level CPB parameters, wherein determine the CPB removal time of the first DU comprises determine the CPB removal time of the first DU based at least in part on the decoded duration and the sub picture level CPB parameters.

29. The computer-readable storage medium of claim 25, wherein the instructions further cause the processor to decode one or more picture level CPB parameters, wherein determine the CPB removal time of the first DU comprises determine the CPB removal time of the first DU based at least in part on the decoded duration and the picture level CPB parameters.

30. A video decoding device comprising: means for decoding a duration between a coded picture buffer (CPB) removal time of a first decoding unit (DU) in an access unit (AU) and a CPB removal time of a second DU, wherein the second DU is subsequent to the first DU in decoding order and in the same AU as the first DU; means for determining the CPB removal time of the first DU based at least in part on the decoded duration; and means for decoding video data of the first DU based at least in part on the CPB removal time.

31. The video decoding device of claim 30, wherein the second DU is one of immediately subsequent to the first DU in the AU in decoding order and a last DU in the AU in decoding order.

32. The video decoding device of claim 30, further comprising means for decoding one or more sub-picture level CPB parameters, wherein means for determining the CPB removal time of the first DU comprises means for determining the CPB removal time of the first DU based at least in part on the decoded duration and the sub-picture level CPB parameters.

33. The video decoding device of claim 30, further comprising means for decoding one or more picture level CPB parameters, wherein means for determining the removal time of the first DU comprises means for determining tire removal time of the first DU based at least in part on the decoded duration and the picture level CPB parameters.

34. A method for encoding video data, the method comprising: determining, for an access unit (AU) including a first decoding unit (DU), a coded picture buffer (CPB) removal time of a second DU, wherein the second DU is subsequent to the first DU in decoding order and in the same AU as the first DU; determining a duration between a CPB removal time of the first DU and the determined CPB removal time of the second DU; and encoding the determined duration.

35. The method of claim 34, wherein the second DU is immediately subsequent to the first DU in the AU in decoding order.

36. The method of claim 34, wherein the second DU is a last DU in the AU in decoding order.

37. The method of claim 34, wherein encoding the determined duration includes encoding the determined duration as one or more sub-picture level CPB parameters.

38. The method of claim 37, wherein encoding one or more sub-picture level CPB parameters comprises: encoding a sub-picture timing supplemental enhancement information (SEI) message that is associated with the first DU.

39. The method of claim 38, wherein the second DU is a last DU in the AU in decoding order, and wherein encoding the determined duration as one or more subpicture level CPB parameters further comprises encoding the determined duration in the sub-picture timing SET message.

40. The method of claim 37, further comprising: encoding a sequence level flag to indicate presence of the sub-picture level CPB parameters either in picture timing SEI messages or in sub-picture timing SEI messages.

41. The method of claim 34, wherein the AU has a Temporalld less than or equal to 0, the method further comprising: encoding at least one of a buffering period supplemental enhancement information (SEI) message or a recovery point SEI message associated with the AU.

42. The method of claim 34, wherein the first DU comprises a non-video coding layer (VCL) network abstraction layer (NAL) unit with nal_unit_type equal to UNSPECO, EOS_NUT, EOB_NUT, in the range of RSV_NVCL44 to RSV NVCIA7 or in the range of UNSPEC48 to UNSPEC63.

43. The method of claim 34, further comprising: deriving at least one of a CPB arrival time and a CPB nominal removal time for the AU at both an access unit level and a sub-picture level regardless of a value of a syntax ele ent that defines whether the first DU is the AU.

44. The method of claim 34, wherein encoding the determined duration includes encoding the determined duration as one or more picture level CPB parameters.

45. The method of claim 44, wherein encoding one or more sub-picture level CPB parameters comprises: encoding a picture timing supplemental enhancement information (SEI) message that is associated with the first DU.

46. A video encoding device comprising a video encoder configured to: determine, for an access unit (AU) including a first decoding unit (DU), a coded picture buffer (CPB) removal time of a second DU, wherein the second DU is subsequent to the first DU in decoding order and in the same AU as the first DU; determine a duration between a CPB removal time of the first DU and the determined CPB removal time of the second DU; and encode the determined duration.

47. The video encoding device of claim 46, wherein the second DU is immediately subsequent to the first DU in the AU in decoding order.

48. The video encoding device of claim 46, wherein the second DU is a last DU in the AU in decoding order.

49. The video encoding device of claim 46, wherein encode the determined duration includes encoding the determined duration as one or more sub-picture level CPB parameters.

50. The video encoding device of claim 49, wherein the video encoder is further configured to encode one or more sub-picture level CPB parameters by encoding a subpicture timing supplemental enhancement information (SEI) message that is associated with the first DU.

51. The video encoding device of claim 50, wherein the second DU is a last DU in the AU in decoding order, and wherein encode the determined duration as one or more sub-picture level CPB parameters further comprises encoding the determined duration in the sub-picture timing SEI message.

52. The video encoding device of claim 46, the video encoder further configured to encode a sequence level flag to indicate presence of the sub-picture level CPB parameters either in picture timing SEI messages or in sub-picture timing SEI messages.

53. The video encoding device of claim 46, wherein the AU has a Temporalld less than or equal to 0, the video encoder further configured to encode at least one of a buffering period supplemental enhancement information (SEI) message or a recovery point SEI message associated with the AU.

54. The video encoding device of claim 46, wherein the first DU comprises a non video coding layer (VCL) network abstraction layer (NAL) unit with nal_unit_type equal to UNSPECO, EOS__NUT, EOB_NUT, in the range of RSV_NVCL44 to RSV_NVCL47 or in the range of UNSPEC48 to UNSPEC63.

55. The video encoding device of claim 46, the video encoder further configured to: derive at least one of a CPB arrival time and a CPB nominal removal time for the AU at both an access unit level and a sub-picture level regardless of a value of a syntax element that defines whether the first DU is the AU.

56. The video encoding device of claim 46, wherein encode the determined duration includes encoding the determined duration as one or more picture level CPB parameters.

57. The video encoding device of claim 46, wherein the video encoder is further configured to encode one or more sub-picture level CPB parameters by encoding a picture timing supplemental enhancement information (SEI) message that is associated with the first DU.

58. A computer-readable storage medium having stored thereon instructions that, when executed, cause a processor of a device for encoding video data to: determine, for an access unit (AU) including a first decoding unit (DU), a coded picture buffer (CPB) removal time of a second DU, wherein the second DU is subsequent to the first DU in decoding order and in the same AU as the first DU; determine a duration between a CPB removal time of the first DU and the determined CPB removal time of the second DU; and encode the determined duration.

59. The computer-readable storage medium of claim 58, wherein the second DU is immediately subsequent to the first DU in the AU in decoding order.

60. The computer-readable storage medium of claim 58, wherein the second DU is a last DU in the AU in decoding order.

61. The computer-readable storage medium of claim 58, wherein encode the determined duration includes encode the determined duration as one or more sub-picture level CPB parameters.']